Computational learning and discovery

Transcription

1 Computatoa earg ad dscover CSI 873 / MAH 689 Istructor: I. Grva Wedesda 7:2-1 pm

2 Gve a set of trag data 1 1 )... ) { 1 1} fd a fucto that ca estmate { 1 1} gve ew ad mmze the frequec of the future error. - marg

3 based o fudametas of statstca earg theor Vapk-Chervoeks theor) X Y=fX) Y Istead of detfg the ukow fucto what cassca statstcs does) the ma goa of VC theor s to mtate the ukow fucto. he ke dscover of VC theor: wo ad o two factors are resposbe for geerazato: -Oe emprca oss) defes how we the fucto appromates data -Aother capact VC dmeso) defes the dverst of the set of fuctos from whch oe chooses a appromato fucto If VC dmeso s fte the oe ca acheve a good geerazato. If t s ot fte the geerazato s mpossbe.

4 Eampes he VC dmeso of ear dcator fuctos s equa I ) 1 sg wb) w

5 Eampes he VC dmeso of the set of fuctos s ft I 1 1 ) sgs a) w

6 Let the vector beog to a sphereof radus R.he theset of - marg separatg hperpaes has a VC dmeto bouded as foows VC dm 2 R m 2 1

7 Let the vector beog to a sphereof radus R.he theset of - marg separatg hperpaes has a VC dmeto bouded as foows VC dm 2 R m marg

8 heorem. Wth probabt 1 oe ca assert that theprobabt that a test eampe hperpaehas P error where m 2 VCdm VC 4 m s the umber of b the - marg as foows 1 2 w ot be separated correct b the - marg the boud dm 1 4m 1) 4 trag eampes hperpaead VC dm that are ot separated correct the VC dmeto bouded 2 R m 2 1

9 - marg

10 Suppose that the data 1 1 )... ) { 1; 1} ca be separated b a hperpae w ) b - marg

11 Bue dots: w ) b 1 Red dots: w ) b 1 Combed: w ) b Varabes: w ad b - marg w ) b w 1

12 Bue dots: w ) b 1 Red dots: w ) b 1 Combed: w ) b Varabes: w ad b - marg w ) b w 1

13 Mamze the marg: ma s.t. w ) b Varabes: w ad b 2 2 w w marg w ) b w 1

14 w b ) 1 or: w ) b 1 w w/ w 1/ Mamze the marg: m w s.t. w ) b 1 Varabes: w ad b

15 Mamze the marg: m w 2 s.t. w ) b1 Varabes: w ad b

16 Mamze the marg: m ww) s.t. w ) b1 Varabes: w ad b

17 Mamze the marg: m.5 ww) s.t. w ) b1 Varabes: w ad b No separabe case: Mamze the marg: s.t. Varabes: m.5 w w) C w ) b 1 wb ad 1

18 b w C w w ) s.t. ) m.5 1 Prma probem ) ) Dua probem C s.t. ).5 ma Varabes: ad wb Varabes:

19 Keres

20 C K s.t. ).5 ma Optmzato probem for fdg support vectors d K ) ) Keres 2 ep ) K Pooma mache: A rada bass fucto mache:

21 Optmzato probem for fdg support vectors ma s.t C K ) Decso rues wth a kere usg foud f f 1 1 K 1 b K )) b K f )) b there s the s bue )) for o the s red some : C C) such crease C ad tra aga

22 that correspod to postve O thesupport vectorscarr he correspod Let I { : α are caed portat formato!!! to the actve costrats of }be theset of support vectors thesupport vectors!!! the prma probem!!! Decso rues usg o the support vectors f f b I I K K I )) b the s bue )) b the s red K )) for f there some : C C) s o such crease C ad tra aga

23 C K s.t. ).5 ma Optmzato probem for fdg support vectors Ce e M s.t. m.5 e K M 1) 1 ) ) where 1 Matab QP settg

24 Lear Prcpe Compoet Aass = Sguar Vaue Decomposto of X X UDV X U D V X U r D V r XV UD

25 Lear Prcpe Compoet Aass = Sguar Vaue Decomposto of X X UD L V L X U r D V r X U r D V r L L L X UD V X

26 Lear Prcpe Compoet Aass = Sguar Vaue Decomposto of X X UD L V L X U D L r UD L V L r XV L X L U D L r X V L r X L r

27 SVM testg wth PCA 1. Cacuate the SVD: X UDV 2. Reduce the dmesoat of the feature space: X X V L trag L testg L cacuated 3. Perform the SVM as usua s UD X L testg for trag data V L for testg data o the trag data