Learning High-Dimensional Data with Artificial Neural Networks. Université catholique de Louvain (Belgium) Machine Learning Group

Transcription

1 Lg Hgh-l h Al Nl Nk Ué hlq Lv (Blgm) h Lg Gp hp://..l..b/mlg/ Ol Hgh-l (H) L & l l l C ly, mpy p phm Spzg C El m l I N ml l m (l) vbl l pj ph: l, SV,

2 Ol Hgh-l (H) L & l l l C ly, mpy p phm Spzg C El m 3 l I N ml l m (l) vbl l pj ph: l, SV, H! Eh q pbl my H! l - lg - g 0.60 k m 0.50 H I = lg Ambl py lvl P lhl +/- 4

3 H! Eh q pbl my H! l - lg - g H lg I = Fm B. Fl & hp://gyl..j. H! Eh q pbl my H! l - lg - g Sp H 6 K y=?,, I+ (, ) = K y, I + 3

4 G ly mb vbl Wh H Wh I myl bl h y m m Spkg, l b. bl L 65 m mb bv Aly l 7 Cly P Clz SUPERVISE UNSUPERVISE C 0?? C 9?? C 7?? H C C C 3 5?????? 8?????? 4

5 Cly P Clz SUPERVISE UNSUPERVISE C 0?? C 9?? H C C 5 C 3 C 7 hgh-l bj???????? 9?????? hk mhml l = R b m v p (El p) l m lgb & gmy H 0 Ppl Cmp Aly k-n Nghb 5

6 h l g l l g H l... N S g, p, Sg h h l bhv C h #bv >> hl hgh-l = 00 = 700 = P g Spl ly V g H = gz = NA q ly = F 6

7 L l Ppl mp ly (PCA): b v m hg (I I) ply h mb l Oh mh: L ly (LA) Pl l q (PLS) Sl pblm! 3 Nl l y = I,, z(θ) K (,,,θ ), θ l m h mz -v l ll m ml pblm (l, hgh lp) vg E: l-ly ly pp, G m (RBF), l-gzg mp,. 4 7

8 I p, lyg H vy l W l h Sm ly h hyph Pm h l llp l I h llg: Wh h l? H l? 5 Ol Hgh-l (H) L & l l l C ly, mpy p phm Spzg C El m 6 l I N ml l m (l) vbl l pj ph: l, SV, 8

9 C ly () () y l l () ml ml () OK KO 7 C ly y l l Nmb p g I = I = I hgh I: v gh I = 3 v pl plly h I () 8 9

10 C ly y l l Empl: Slvm (986) Nmb G kl y pp (G) b I # kl E+06 E+05 E+04 E+03 E+0 E+0 E I 9 Empy p phm Sl v h m pblm: m h p p y p m l pm : v m G b I ( ) ) my Empl: hgm I mpmz b y # h b 0 0

11 Spzg : ph Vlm ph (=) I I= I= g z p l vlm vl = I V vl = π ( ) = Γ π / ( ) / Spzg : R vlm ph / b vlm g z p l H, ll p h h I

12 Spzg : ph Vlm mb ph g z p l 0.9 vlm I 3 Spzg : G l-i G b I= g z p l % p ph.65 % p I 4

13 Spzg : G Ah v hgh-i G b: Pbbly p m h I-l mlml b g z p l 5 C El m m l 6 b h m m v... mp [0,] m [ 0, I ] = = 50 Nm h p 3

14 h m hl = = 00 m l 7 P m ly ql ll p Rl vh h I l E ( X ) ( X ) V h = 0 h AX IN IN p 0 [By] C El m m l m m l m m = m El : lv m ly? 0 8 [By] 4

15 Ol Hgh-l (H) L & l l l C ly, mpy p phm Spzg C El m 9 l I N ml l m (l) vbl l pj ph: l, SV, I l Empy p phm Ny h Sl Pj Pbl b 30 5

16 I l 3 ll PCA b g mll g pply pl PCA ppl mll g p Gbg-P P I l ll PCA b g p Gbg-P P P. m 3 6

17 I l ll PCA b g Sl b g p Gbg-P P l b p p Avg: N(N-)/ N p lg N lg gph: mb N p p l h 33 I l IN ll PCA b g Nl k mpl: g pblm p Gbg-P P OU Pm OK? 34 7

18 I : l h p l S m vy l S m vy 35 Ll lg g l l l l «Ll»:: by m ll ypl mpl: G kl Rl B F Nk = P j K j j = ( ) ( ) K ( ) Ll glbl: ANN pl, pl g h lg! j = p j h j 36 8

19 Rl-B F Nk (RBFN) pp g l l l l = P j K j j = ( ) ( ) K ( ) Avg (v LP, ): pl mp lg j = p j h j - j - h h j - mlplyg j 37 Pblm h (ll?) lg ANN b p gh v: g l l l l 38 RBFN: VQ SO: ly LP: gm l kl h h I hgh-l p: ll h m l! ( m phm) h l k m l k, h h m 9

20 Ug m l l Empl: RBF U «p-g» kl: ( ) = P ( ) K j ( ) j K j j = = p j h j kl: K ( ) j = p j h j 39 N mg v l l h kl RBF ly ql h SV mpl h V z! j σ j = SV p K j ( ) = p ( V z) qσ j j RBF q 40 0

21 l l 4 kk m gh X [ X, ] R =, L X p =,,... p > R p > 0 R p = p = p = p X = p p X = kk m Fl m Fl p-m h m El m y m = = = m Fl m pbl l l h q vl y p p pv h mp mly b. F g, l m l h hg- m E ( ) p ( ) V p p = /4 p = / p = p = 4 [Aggl] 4

22 Sm C Q l l. Fl m ll h gl qly. hv q?. ll m? 3. Shl h p b mll pbl? I h l l lly? 43 Sm C Q l l. Fl m ll h gl qly. hv q? G q. ll m? Y. B hgh hgh- m. 3. Shl h p b mll pbl? N. h mll h p, h mll h b. 44

23 Fl Nm m h hgh- m l l Pp NN qy hh g p = /4 p = / p = p = 4 45 l Bk h gl pblm I ' LOWER R h ly Op l lp hm ' ()l ml y 46 3

24 Op lp l I Sl Pj mpl mh ' ' LOWER L Cl ()( PCA ()l ml Nl y l m ()( Smm mppg & vlpm ( )( 47 Cl lp l I mpl mh Sl Pj ' ' L LOWER Cl (( y) LA, PLS ()l ml Nl h, g, gy (-bk) y l m (( y) 48 4

25 Nl pj l 49 Smm mppg ( X Y ) X j N N E = / j j = j = gl p pj p h p lg Nl pj l N vlpm N N N N = ( X Y ) X ( ) ( ) j E X Y F Y, λ E / j j = j = = = j = lg (mh: CCA, CA,.) U vl = h b j j j 50 5

26 Cvl pj l 5 H pj l ml g g?, J.A. L,, ESANN 004. Appl g k l Rg:- p vl (-) Fg - g (j) K K ( + ) = ( (), ( ),, ( k ), (), (), () l ), N-l g:. p g l p. h m g h -l p l (mpl l!) 5 6

27 Fg: l p vbl l Sg h my p vbl, h h mb p:. l p vbl pbly l g vbl. pj p vbl l: PCA -l: CCA, Kh,,. b k hm 53 Fg: k hm / () (-) l k hm: (q) = q z g q+ (AR ml) (-) () () (-) 54 7

28 Fg: k hm / l k hm: q z g q+ I h q+ p, h q- h p Pj m q+ q pbl! 55 Fg: mpl / Al l 56 ( + ) = () + b( ) + ε() p vl! L AR ml Sm ( 000 mpl) AR 8

29 Fg: mpl / l N-l AR ml l g: : z=6 : CCA m =6 = LP - Sm ( 000 mpl) LP h pj -l pj AR 57 Fg: mpl / l 58 ly BEL0 4 m p g vbl: :, -0, -0, -40,, y, y -0, : - -5, -5 -,, y -y -0-5 ll: K(0), K(40), mvg vg: (0), (50), pl mvg vg: E(0), E(50),

30 Fg: mpl / h: l 59 4 PCA 5 vbl CCA 9 vbl RBF g Rl: % pp g (90-y vg) I vg: 57.% Gbg-P P: : = F ph Fl ly h p l Ully: g vbl lv B: p

31 Fl ly C (mh) h p l ,, K, I g y = (,, K,,θ) y = ( g,θ) I 6 F ph Spp V h (SV) h p l 6 N mhml ml l l Avg: l ml: mply l h # ( h I) ll mm ( p h pblm, h kl,.) bk Ch h kl pm Why hl h mpy p phm pp? 3

32 Cl H ly: y, b y Sm phm m blm; h m p? Aly... h ly R ly (l, pj) Ap h l U l m U h 63 hk All mmb h h Lg Gp hp://..l..b/mlg/ I pl m F (l m) Jh L (-l pj) Gy S ( ) Nl ly (l ly) A ll h llg kg (H), v hy y 64 3