Reproducing Kernel Hilbert Space for. Penalized Regression Multi-Predictors: Case in Longitudinal Data

Transcription

1 Interntonl Journl of Mthemtcl Anlyss Vol. 8, 04, no. 40, HIKARI Ltd, Reproducng Kernel Hlbert Spce for Penlzed Regresson Mult-Predctors: Cse n Longudnl Dt Adj Achmd Rnldo Fernndes Deprtment of Mthemtcs Fculty of Mthemtcs nd Nturl Scences Unversy of Brwjy Jln Vetern Mlng-Indones I Nyomn Budntr, Bmbng Wdjnrko Otok nd Suhrtono Deprtment of Sttstcs Fculty of Mthemtcs nd Nturl Scences Sepuluh Nopember Instute of Technology Jln Arf Rhmn Hkm Surby-Indones Copyrght 04 Adj Achmd Rnldo Fernndes, I Nyomn Budntr, Bmbng Wdjnrko Otok nd Suhrtono. Ths s n open ccess rtcle dstrbuted under the Cretve Commons Attrbuton Lcense, whch perms unrestrcted use, dstrbuton, nd reproducton n ny medum, provded the orgnl work s properly ced. Abstrct Penlzed regresson procedures hve become very populr pproch to estmtng the curve of nonprmetrc regresson n longudnl dt. Reproducng Kernel Hlbert Spce (RKHS) s Reproducng Kernel Hlbert Spce (RKHS) ply centrl role n Penlzed Regresson s form nd estmtor functon of the model. The m of ths study re to solve the estmton of Penlzed Regresson usng RKHS, nd pply the Penlzed Regreson usng secondry dtset. The Penlzed Regressson usng RKHS s f ( x) T d V c, The estmton of ˆf T dˆ V cˆ A y wh: ' A = T T M WT T ' M W + VM W[ I T T ' M WT T ' M W ].

2 95 Adj Achmd Rnldo Fernndes et l. The plcton of dt results show tht the splne estmtor cn be ppled to the generton of dt wh m = 4 (cubc splne) whch gves the vlue of R of 97.77%. Keywords: Penlzed Regresson, RKHS, Longudnl, Mult-predctor Introducton Penlzed regresson procedures hve become very populr pproch to estmtng the curve of nonprmetrc regresson,.e splne estmtor [3,6]. One of the uses of regresson nlyss s n the nlyss of longudnl dt, whch s combnton of cross-secton dt nd tme-seres, tht s the observtons whch re mde s mny s r mutully ndependent subjects (cross-secton) wh ech subject s repetedly observed n n perod of tme (tme-seres) nd between observtons whn the sme subjects whch re correlted [,5]. In longudnl dt ( x, x,..., xp, y ), the reltonshp between the mult-predctor vrbles wh sngle-response vrble follows the regresson model cn be presented s follows: p () y f ( x ),,,...,p;,,..., r; t,,..., n. Equtons () s the regresson model for longudnl dt wh mult-predctor vrbles, nd f s the regresson curve reltonshp between the predctor vrbles wh the response vrble y for to- subject. The curve f s unknown functon n non-prmetrc regresson pproch [3]. Reproducng Kernel Hlbert Spce (RKHS) ply centrl role n Penlzed Regresson s form nd estmtor functon of the model [6]. Bsed on the bove bckground, the purposes of ths study re () to solve the estmton of Penlzed Regresson usng Reproducng Kernel Hlbert Spce, nd () to pply the Penlzed Regreson usng dt of decubus wound from Fernndes, et l. [4]. Mterls nd Methods. Interpoltng Splnes Suppose we wnt to fnd functon f tht nterpoltes between the ponts ( x, ),,,...,p;,,..., r; t,,..., n nd the boundry vlue of x x... x n b The functons fw where m W m { f : f sbsolutelycontnuouson[, b], f ' L [0,]}, () nd we clled n Sobolev Spce [6]. If we restrct the boundry vlue n 0 x x... x n wh trnsformton, then the vlue of f (0) 0s not relly necessry, but smplfes the presentton of dervtve vlue of f ' or f '. Defnng n nner product of

3 Reproducng kernel Hlbert spce 953 W m by: f, g f '( x) g '( x) dx, (3) 0 Imples norm over the spce W m tht s smll for smoth functons [6]. To ddress the nterpolton problem, s gven: f, R f '( x) R' ( x) dx f( x ) f(0) f( x ). (4) Thus nterpoltor 0 f, stsfes system of equton (5), nmely: f ( x ) R, f,,...,p;,,..., r; t,,..., n, (5) nd the smoothest functon The p r n t f stsfes n equton (6) f ( x ) R ( x) (6) s re the solutons to the system of rel lner equtons obtned by substutng of f ˆ stsfes nto (5), p r n t R, R s Note tht R, R R ( x ) R ( x ) mn( x, x ) (8) s s t s s nd defne the functon R ( x ) mn( x, x ) whch turns out to be reproducng kernel. xs t s t. Penlzed Weghted Lest Squre The splne pproch generlly defnes fk n equton () n form of n unknown regresson curve, but f s only ssumed s smooth, n sense of beng contned n specfed functon spce, especlly Sobolev spce n equton (). Optmzton Penlzed Weghted Lest Squre (PWLS) nvolves weghtng n form of rndom error vrnce-covrnce mtrx W. To obtn the estmte of the regresson curve fk usng optmzton PWLS tht s the completon of optmzton s follows [3]: (7) r ( m) Mn N ( y f )' W ( y f ) ( f ( x )) dx. (9) 0 m fw [0,] The PWLS optmzton n equton (9) n ddon to consderng the weght, lso consders the use of r smoothng prmeter s controller between the goodness of f (the frst segment) nd the roughness penlty (second segment).

4 954 Adj Achmd Rnldo Fernndes et l. 3 Result nd Dscusson 3. Reproducng Kernel Hlbert Spce Hlbert spces tht dsply certn propertes on certn lner opertors re RKHS. The functon f the unknown functon nd ssumed smooth n the sense of beng contned n the spce H. Then the spce H s decomposed nto drect sum of two spces H0 nd H, tht s H H0 H, wh H0 H. Suppose the bss for the spce H0 s,,, m nd the bss for the spce H s,,, n, then for ech functon f H cn be presented ndvdully s: f g h, (0) Furthermore, for every functon H cn be presented ndvdully s : f g h m d + j j j f n c = t + c () ' d L x s lmed lner functon n the spce H nd functon f H, obtned L f L ( g h ) f ( x ) () x x Bsed on the Resz Representton Theorem nd L x s lmed lner functon n the spce H, obtned sngle vlue H whch s the representtve of L x, nd completes the equton : L f, f f ( x ), f H (3) x Bsed on the equton (0), then fk ( x ) n the equton (3) cn be expressed s : f ( x ) =, ' d, c (4) The descrpton of the equton (4) for sngle predctor ( ) nd k,, obtned : f ( x ), ' d, c, t,,, n. t t t f( x) f( x) f( x) f ( x ) d, d, c, c, d, d, c, c, m m n n m m n n n d n, d m n, m c n, c n n, n,,, m d,,, n c,,, m d,,, n c,,, d,,, c n n n m m n n n n n f ( x ) Td V c (5)

5 Reproducng kernel Hlbert spce 955 Then from the sme wy, ws obtned the result for =,3,...,r f ( x ) Td V c (6) T s the mtrx of order nm, d s the vector of order m, V s the mtrx of order nn, c s the vector of order n. Thus, the form of penlzed regresson mult-predctors model f( x) s s follows: f ( x) Td V c, (7) T s the mtrx of order (rn)(rm) nd V s the mtrx of order (rn)(rn) s follows: T 0 0 V T 0 T nd 0 V 0 V (8) 0 0 Tr 0 0 Vr Solvng the f bsed on equton of (7) for mult-predctor (,,..., p ) nd p f ( x ) f ( x ). (9) Thus the Penlzed Regressson s gven n equton below: f ( x) T d V c (0) T s the mtrx of order (rn)(rm) s follows: T T T 0, wh 0 0 T r nd V V 0 V, wh 0 0 V r Mtrx T T T T p..., V s the mtrx of order (rn)(rn) s follows: T s reproducng kernel n H0 s follow:,,, m,,, m T n, n, n, m V V.... k V V p (),

6 956 Adj Achmd Rnldo Fernndes et l., L, t,,..., n; j,,..., m. () ( j )! j j Mtrx V j x V s reproducng kernel n H s follow:,,, n,,, n,,, n n n n, b m m ( x u) ( xs u), s L s R ( x, xs ) du. (3) ( m )! m m w ( x u) ( x u) f x u 0 or x u nd ( x ) m u 0 f x u 0 or x u. In ths cse, u b, nd mn( x k ), then x u, so m m ( x u) ( x u). If m = 4 (cubc splne), then b 3 3 ( x u) ( xs u), s du (4) 6 b ( 3 3 )( s 3 s 3 s ) x x u x u u x x u x u u du b {( x xs 3x xsu 3 x xsu xu ) ( 3x xsu 9x xsu 9xxsu 3 xu ) (3 s 9 s 9 s 3 ) ( s 3 s 3 s )} x x u x x u x x u x u x u x u x u u du b 36 {( x x ) 3( x x x x ) u 3( x x 3 x x x x ) u ks s ks s s s ( 9 ks 9 s s ) 3( 3 s s ) 3( s ) )} x x x x x x u x x x x u x x u u du ( x x ) u ( x x x x ) u ( x x 3 x x x x ) u s 4 s s 36 s s s b 44 x x xs x xs xs u 60 x xxks xs u 7 x xs u 5 u ( 9 9 ) ( 3 ) ( ). For x [0,], solvng ths equton s:, ( x x ) ( x x x x ) ( x x 3 x x x x ) s 36 s 4 s s 36 s s s s s s 60 s s 7 s 5 ( x 9x x 9 x x x ) ( x 3 x x x ) ( x x ). (5) For the purposes of ˆf estmton, RKHS pproch wh completes the Penlzed Weghted Lest Squre (PWLS) creron s follow: Mn fh,,.., r Wh constrs : W = Mn ( y f) f H,,.., r f, 0 W, (6) m Then functon H = W [0,] used ws Sobolev spce whch s defned n (). Whb [6] hs solve ths equton s:

7 Reproducng kernel Hlbert spce 957 wh: ˆf T dˆ V cˆ ' = { ' T T M WT T M W + V M W[ I T T ' M WT T M W ]}y. A y (7) M WV NI ' A = T T M WT T ' M W + VM W[ I T T ' M WT T ' M W ] Applcton of Dt The penlzed regrseson mult-predctors model n equton (7) nd solve estmton n equton (7) ws ppled n the dt decubus wound from Fernndes, et l. [4] wh p=3, r=8 nd n=3. The Plot between predctor vrbles x nd response vrble y to be gven to Appendx. From the smulton results, no form of prtculr pttern (the pttern ws less cler form) between the predctor vrbles x wh response vrble y nd subject (yk). The next stge s to choose the smoothng prmeter bsed on the vlue of the mnmum Generlzed Cross Vldton (GCV). Appendx presents the results of the prtl smoothng prmeter ( k ) bsed on the mnmum vlue of GCV. Appendx presents the results of the prtl smoothng prmeter selecton for ech prmeter by condonng the other prmeters whch were constnt. The ooptmztons results showed tht the optmzton of the mnmum vlue of GCV s mny s for ech prmeter vlue s follows: = 0.0 = = = = = = = 0.0

8 958 Adj Achmd Rnldo Fernndes et l. Grph : Penlzed Regresson Mult-predctors Model for Longudnl Dt Grph : (cont ) Grph shows the results of Penlzed Regresson for mult-predctor longudnl dt tht provde the mnmum vlue of GCV. From dt set gven the optml soluton, the curve cn descrbe 97.77% of the vrnce of the orgnl dt. 4 Concluson Bsed on the results of the study presented on the prevous prt, severl thngs cn be concluded s follows:

9 Reproducng kernel Hlbert spce 959. The Penlzed Regressson usng RKHS s T T T T r, wh f ( x) d c T T..., T Tp nd T V, where V V 0 V, wh V V.... k k V k V pk 0 0 V r The estmton of ˆf T dˆ V cˆ A y wh : M WV NI ' A = T T M WT T ' M W + VM W[ I T T ' M WT T ' M W ].. The plcton of dt results show tht the splne estmtor cn be ppled to the generton of dt wh m = 4 (cubc) whch gves the vlue of R of 97.77%. Acknowledgements. Mny thnks to the Drectorte Generl of Hgher Educton, the Mnstry of Educton nd Culture of the Republc of Indones psssed through BPPS grnt nd Unversy of Brwjy for fnncl support. References []. Budntr, I.N., Subnr nd Soejoet, Z. (997). Weghted Splne Estmtor. Proc 5 st Sesson of the Interntonl Sttstcl Instute, Istnbul, []. Dggle, P.J., Lng, Y.K. nd Zeger, S.L., (006), Anlyss of Longudnl Dt, Second Edon. Oxford Sttstcl Scence Seres 3, New York. [3]. Eubnk, R.L. (999). Nonprmetrc Regresson nd Splne Smoothng. Second Edon. New York. Mrcel Dekker, Inc.

10 960 Adj Achmd Rnldo Fernndes et l. [4]. Fernndes, A.A.R, Budntr, I.N., Otok, B.W., nd Suhrtono (0). Applcton Of Mult-Response Longudnl Dt Usng Lner Mxed Model (Cse Study n Ptent wh Pulmonry Tuberculoss n Syful Anwr Hospl Mlng). Proceedng of nd Regonl Conference on Appled nd Engneerng Mthemtcs (RCAEM-II) Unversy Mlys Perls, 0. [5]. Hrdle, W. (990), Appled Nonprmetrc Regresson. New York. Cmbrdge Unversy Press [6]. Whb, G. (990). Splne Models for Observtonl Dt. Pensylvn. SIAM Appendx. Plot of Smulton Dt

11 Reproducng kernel Hlbert spce 96 Appendx. Smoothng Prmeter Receved: July, 04