STA 414/2104 Mar 11, Notes

Transcription

1 STA 414/2104 Mr 11, / 12 Notes Tke-home Midterm: Mrh 16 Mrh 25 One qestion from Kernels nd Ensemles y M. Zh Clssifition nd regression trees 9.2 Ensemle methods nd rndom forests Zh k-mens nd k-nerest neighor methods nspervised lerning

2 STA 414/2104 Mr 11, / 12 VR ode for nerl networks Size = 2 Size = 2, lmd = Pregnnetriol Pregnnetriol Tetrhydroortisone Tetrhydroortisone Size = 2, lmd = 0.01 Size = 5,20, lmd = 0.01 Pregnnetriol Pregnnetriol Tetrhydroortisone Tetrhydroortisone

3 STA 414/2104 Mr 11, / 12...ode for nn plt.ndry <- fntion(size=0, dey=0,...) { sh.nn <- nnet(sh, tpi, skip=t, softmx=t, size=size, dey=dey, mxit=1000) invisile(1(predit(sh.nn, sht),...)) } 1 <- fntion(z,...) { zp <- Z[,3] - pmx(z[,2], Z[,1]) ontor(exp(xp), exp(yp), mtrix(zp, np), dd=t, levels=0, lex=0,...) zp <- Z[,1] - pmx(z[,3], Z[,2]) ontor(exp(xp), exp(yp), mtrix(zp, np), dd=t, levels=0, lex=0,...) }

4 STA 414/2104 Mr 11, / 12...ode for nn Z = predit(sh.nn, sht) ses fitted nerl network to predit on grid of (x 1, x 2 ) lled sht the grid points re lled xp nd yp this gives 10, mtrix of proilities zp = Z[,3] - pmx(z[,2], Z[,1] ) Pr(3) mx{pr(2), Pr(1)} positive if lss 3 is the highest proility, else negtive mtrix(zp, np): np= 100, the nmer of olmns in the mtrix zp ontor(exp(xp), exp(yp), mtrix(zp, np), dd=t, levels = 0) the ondry where Pr(3) is lrgest nd similrly zp = Z[,1] - pmx(z[,3], Z[,2] ) Pr(1) mx{pr(3), Pr(2)} positive if lss 1 is the highest proility, else negtive ontor(exp(xp), exp(yp), mtrix(zp, np), dd=t, levels = 0: the ondry where Pr(1) is lrgest

5 STA 414/2104 Mr 11, / 12 Spport Vetor Mhines nder onstrints 1 min β 0,β 2 β 2 + γ ξ i 0, i=1 N ξ i s.t. ξ i 0 i=1 y i (x T i β + β 0 ) 1 ξ i Lgrnge mltipliers: L P = 1 N N 2 β 2 +γ ξ i α i {y i (xi T β+β 0 ) (1 ξ i )} minimize over β, β 0, ξ i i=1 ˆβ = {x i : ˆα i 0} spport vetors N ˆα i x i y i i=1 N µ i ξ i i=1

6 STA 414/2104 Mr 11, 2010 lirry(e1071) lirry(rgl) Code for Spport Vetor Mhines Jen-Frnçois Plnte, U de M ## similr ondry drwing fntions for ## nerl networks et re in Chpter 12 sripts for MASS ondries=fntion(y,,n=100){ # y is the dt, Ê the svm ojet grid=expnd.grid(seq(min(y[,1]),mx(y[,1]),length=n),seq(min(y[,2]),mx(y[,2]),length=n)) points(grid,ph=4,ex=.15,ol=pste(s.hrter(predit(,grid)))) } spports=fntion(y,){ # y is the dt, Ê is the svm ojet points(y[$index,],ex=2) } # Exmples of SVM # y=mtrix(rnorm(200),nol=2)+rep((0,5),eh=50) l=rep(("red","ble"),eh=50) plot(y,ol=l,ph=20) =svm(y,s.ftor(l),kernel="liner") smmry() spports(y,) ondries(y,) 6 / 12

7 STA 414/2104 Mr 11, / 12 Exmple with mislssifition y=mtrix(rnorm(200),nol=2)+rep((0,2),eh=50) l=rep(("red","ble"),eh=50) plot(y,ol=l,ph=20) =svm(y,s.ftor(l),kernel="liner") spports(y,) smmry() fit=s.hrter($fitted) plot(y,ol=l,ph=20) points(y[fit!=l,],ol=fit[fit!=l],ph=5,ex=1.5) ondries(y,)

8 STA 414/2104 Mr 11, 2010 Exmple with irles norm=fntion(x){sqrt(sm(xˆ2))} y=mtrix(rnorm(200),nol=2) y[51:100,]=y[51:100,]+4*y[51:100,]/pply(y[51:100,],1,norm) l=rep(("red","ble"),eh=50) plot(y,ol=l,ph=20) =svm(y,s.ftor(l),kernel="liner") smmry() fit=s.hrter($fitted) points(y[fit!=l,],ol=fit[fit!=l],ph=5,ex=1.5) ondries(y,) =svm(y,s.ftor(l),kernel="rdil") smmry() fit=s.hrter($fitted) plot(y,ol=l,ph=20) points(y[fit!=l,],ol=fit[fit!=l],ph=5,ex=1.5) ondries(y,) plot(y,ol=l,ph=20) points(y[fit!=l,],ol=fit[fit!=l],ph=5,ex=1.5) spports(y,) # Chek new dt y2=mtrix(rnorm(200),nol=2) y2[51:100,]=y2[51:100,]+4*y2[51:100,]/pply(y2[51:100,],1,norm) l=rep(("red","ble"),eh=50) fit=s.hrter(predit(,y2)) plot(y2,ol=fit,ph=20) points(mtrix(y2[fit!=l,],nol=2),ex=1.2) 8 / 12

9 STA 414/2104 Mr 11, / 12 Exmple with mny lsters norm=fntion(x){sqrt(sm(xˆ2))} y=mtrix(rnorm(400),nol=2) y[51:100,]=y[51:100,]+3*y[51:100,]/pply(y[51:100,],1,norm) y[101:150,]=y[101:150,]+(4,4) y[151:200,]=y[151:200,]+6*y[151:200,]/pply(y[151:200,],1,norm) l=rep(("red","ble","green","blk"),eh=50) plot(y,ol=l,ph=20) hk=smple(1:200,20) =svm(y[-hk,],s.ftor(l[-hk]),kernel="rdil") smmry() ondries(y,) spports(y,) fit=s.hrter(predit(,y[hk,])) plot(y[-hk,],ol=l[-hk],ph=20) points(y[hk,],ol=fit,ph=3) # Points with wrong tegory points(mtrix(y[hk[fit!=l[hk]],],nol=2),ol=l[hk[fit!=l[hk]]],ph=7)

10 STA 414/2104 Mr 11, 2010 Exmple with msi files # Rel dt. List of songs. Vriles re sed on the nlysis # of the signl of 30 seonds of msi. # Use tht dt to predit the type of songs. =red.tle("msi_dt_17_10_05.ex",sep=" ")[,-52] x=rry(s.nmeri(s.mtrix([,2:50])),dim()) rem=pply(is.n(x),1,sm) tit=[!rem,1] genre=[!rem,51] x=x[!rem,] x=x[,pply(x,2,sd)!=0] # Visliztion pirs(x[,smple(1:46,5)],ol=l,ph=20) keep=smple(1:46,3); kp=pste("vr",keep); plot3d(x[,keep],size=3,ol=l,xl=kp[1],yl=kp[2 # Remove dt for vlidtion hk=smple(1:1559,200) # Different SVM =svm(x[-hk,],genre[-hk]) sm(predit(,x[hk,])==genre[hk]) 1=svm(x[-hk,],genre[-hk],kernel="liner") sm(predit(1,x[hk,])==genre[hk]) 10 / 12

11 STA 414/2104 Mr 11, / 12 Vry the vle of C /γ 1=svm(x[-hk,],genre[-hk],kernel="liner") sm(predit(1,x[hk,])==genre[hk]) 1=svm(x[-hk,],genre[-hk],kernel="liner",ost=10) sm(predit(1,x[hk,])==genre[hk]) 1=svm(x[-hk,],genre[-hk],kernel="liner",ost=.1) sm(predit(1,x[hk,])==genre[hk]) 1=svm(x[-hk,],genre[-hk],kernel="liner",ost=100) sm(predit(1,x[hk,])==genre[hk]) # Dt from the ook x=mtrix(sn("x.dt"),nol=2) y=sn("y.dt") y=("ornge","le")[2-y] plot(x,ol=y,ph=20) =svm(x,s.ftor(y)) ondries(x,) spports(x,) =svm(x,s.ftor(y),kernel="polynomil",degree=4) ondries(x,) spports(x,)

12 STA 414/2104 Mr 11, 2010 Msi files The Grgend Fetre Set The dt files ontin fetres for most of the dio trks in Grgend dtset. These hve een extrted sing the pproh desried in [Miersw/Morik/2005]. More informtion ot the Grgend dtset nd the dio trks n e fond in [Homrg/etl/2005]. Formt The dtset is represented in the Yle formt nd n e red nd proessed with the Yle pkge ( History We hve revised the former exmpleset, whih n e fond nder " We repled ".mp3" with "_mp3" in the sffixes of the exmple ids, to mke them omptile with the ser txonomies. Also 30 exmples were removed to provide onsisteny with nother exmpleset ontining more fetres whih will e ville soon. Referenes [Homrg/etl/2005] Homrg, Helge nd Miersw,Ingo nd Möller, Bülent nd Morik, Kthrin A Benhmrk Dtset for Adio Clssifition nd Clstering. In Pro. of the Interntionl Symposim on Msi Informtion Retrievl 2005, [Miersw/Morik/2005] Miersw, Ingo nd Morik, Kthrin. Atomti Fetre Extrtion12 for / 12 Cl