SCITECH Volume 5, Issue 1 RESEARCH ORGANISATION November 17, 2015

Transcription

1 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: SCITECH Volum 5, Issu RESEARCH ORGANISATION Novmbr 7, 5 Journal of Informaton Scncs and Computng Tchnologs A nw approach calld Wghtd Last Squars Rato (WLSR) Mthod to M-stmators Murat Yazc JForc Informaton Tchnologs Inc. Göztp mah. Göksuvlr St. Sardunya Sk. BB Istanbul 3485, Turky Abstract Rgrsson Analyss (RA) s an mportant statstcal tool that s appld n most scncs. Th Ordnary Last Squars (OLS) s a tradton mthod n RA and thr ar many rgrsson tchnqus basd on OLS. Th Wghtd Last Squars(WLS) mthod s tratvly usd n M-stmators. Th Last Squars Rato (LSR) mthod n RA gvs bttr rsults than OLS, spcally n cas of th prsnc of outlrs. Ths papr ncluds a nw approach to M-stmators, calld Wghtd Last Squars Rato (WLSR), and comparson of WLS and WLSR accordng to man absolut rrors of stmaton of th rgrsson paramtrs (ma ß) and dpndnt valu (ma y). Kywords: Outlrs Last squars rato (LSR) mthod Wghtd last squars rato (WLSR) mthod Robust statstcs M-stmators.. Introducton Th thory of robustnss dvlopd by Hubr and Hampl (96) lad th foundaton for fndng practcal solutons too many problms, whn statstcal concpts wr vagu to srv th purpos. Robust rgrsson analyss hav bn dvlopd as an mprovmnt to last squars stmaton n th prsnc of outlrs and to provd us nformaton about what a vald obsrvaton s and whthr ths should b thrown. Th prmary purpos of robust rgrsson analyss s to ft a modl whch rprsnts th nformaton n th majorty of th data. Robust rgrsson s an mportant tool for analyzng data that ar contamnatd wth outlrs. It can b usd to dtct outlrs and to provd rsstant rsults n th prsnc of outlrs. Many mthods hav bn dvlopd for ths problms. Many rsarchrs hav workd n ths fld and dscrbd th mthods of robust stmators. Th class of robust stmators ncluds M-, L- and R-stmators. Th M-stmators ar most flxbl ons, and thy gnralz straghtforwardly to multparamtr problms, vn though thy ar not automatcally scal nvarant and hav to b supplmntd for practcal applcatons by an auxlary stmat of scal any stmat (Muthukrshnan and Radha, ). Th tratvly ordnary last squars approach s usd n M-stmators durng th calculaton of th rgrsson paramtrs. In ths approach, th wghtd rrors ar calculatd by usng a wghtng functon n ach tratv stp. Employng th wghtng functon n OLS, w gt wghtd last squars (WLS) mthod. Ths papr ncluds a nw approach calld Wghtd Last Squars Rato (WLSR) Mthod to M-stmators as an altrnatv to WLS Mthod. In ths mthod, th rrors ar calculatd by usng th LSR mthod nstad of th OLS mthod. Thn, th wghtd rrors ar calculatd by usng a wghtng functon n ach tratv stp. Volum 5, Issu avalabl at 399

2 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: In ths study, t was shown whch mthod (WLS and WLSR) gvs bttr rsultsn M-stmaton ncludng Hubr, Tuky, Andrw and Ramsay s functonsaccordng to th man absolut rrors (MAE) of th stmatd rgrsson paramtrs and dpndnt valu va a smulaton study usng dffrnt sampl szs and rror varancs. Basd on th smulaton rsults, apart from Andrw s functon, w can say that WLSR gnrally gvs bttr stmats than WLSn cas of ncrasngth numbr of outlrs and th rror varanc.. Th LSR Mthod Th Last Squars Rato (LSR) mthod s on of th forcastng tchnqus n rgrsson analyss. LSR ams to stmat obsrvd valus wth zro rror (Y Y, ory Y ). It starts wth th sam goal Y Y as n Ordnary Last Squars. Howvr, t procds by dvdng through by Y and so YY / s obtand undr an assumpton ofy. Hnc, t s obvous that, quatons ( YY / ) and ( Y Y) / Y ar rasd by basc mathmatcal opratons. Ths fnal quaton s takn nto account as th orgn of th LSR whch mnmzs th sum of[( Y Y) / Y]. Consquntly th am of LSR can b wrttn mathmatcally as follows (Akblgc and Aknc, 9): n Y-Y mn () Y Th matrx rprsntaton of th rgrsson modl s as follows; Y=βX+ () whr Y s an n vctor of obsrvd valus; X s an n p vctor of th valus of dpndnt varabls; n s th numbr of obsrvatons; p s th numbr of unknown paramtrs, β s th p vctor of rgrsson coffcnts; s an n vctor of rror valus. Formula can also b wrttn as n formula 3, by usng Eq. : n lsr X mn (3) LSR Y- Y If rank(x) s qual to p, th formula for stmatng β appars as n Eq. 4 (Akblgc and Aknc, 9): ' ' lsr X X X Y. (4) Y Y Y Th matrx X / Y s obtand by dvdng th valus dvdng th valus x j by y for j,,..., p. x j by y for j,,..., p, and X / Y s computd by 3. M-Estmatorsandth proposdwlsr Mthod Frst proposd by Hubr (964, 973, 4), M-stmaton for rgrsson s a rlatvly straghtforward xtnson of M-stmaton for locaton. It rprsnt on of th frst attmps at a comproms btwn th ffcncy of th last squars stmators and th rsstanc of th LAV stmators, both of whch can b sn as Volum 5, Issu avalabl at 4

3 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: spcal cass of M-stmaton. In smplst trms, th M-stmator mnmzs som functon of th rsduals. As n th cas of M-stmaton locaton, th robustnss of th stmator s dtrmnd by th choc of wght functon (Andrsn, 7). T,..., n X X n for th functon and th sampl x,..., x n s th valu of t that Th M-stmat maxmzs th objctv functon x; t n. Whn can b dffrntatd wth rspct to t, a functon (whch xcpt for a multplcatv constant) w dnot by,.. x ; t mor convnnt to calculat n T by fndng th valu of t that satsfs x t n x ; t t -functon (wght functon) for any s thn dfnd as follows (Al and Qadr, 5); w x ; t, w may fnd t ;. Th corrspondng w x ; t. (5) t Employng ths w -functon n OLS, w gt wghtd last squars (WLS) mthod and th rsultng stmats ar thn calld th wghtd stmats. Th wghtd stmats ar computd by solvng th followng quaton (Hoagln t al., 983); ' X WX ' X Wy (6) whr W s a n x n dagonal squar matrx havng th dagonal lmnts as wghts. Whn w us th w -functon n LSR, w gt wghtd last squars rato.ths mthod s namd as wghtd last squars rato (WLSR) mthod. And, th wghtd stmats ar calculatd by solvng th followng quaton; ' ' X X X WLSR W WY Y Y Y (7) M-stmators mnmz objctv functon mor gnral than th famlar sum of squard rsduals assocatd wth th sampl man. Instad of squarng th dvatons of ach obsrvaton x from th stmat t, w apply th functon x; t ; and form th objctv functon by summng ovr th sampl: x; t of x; t ; dtrmns th proprts of th M-stmator (Hoagln t al., 983). Hubr s M-stmator uss th followng -functon; n. Th natur a, t a t t, a t a a, t a (8) Both th last squars and Hubr objctv functons ncras wthout bound as th rsdual dparts from, but th last-squars objctv functon ncrass mor rapdly. Last squars assgns qual wght to ach Volum 5, Issu avalabl at 4

4 obsrvaton; th wghts for th Hubr stmator dcln whn t Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: a. Th Hubr s -functon taks nto account th nghborhood of a normal modl n a lnar way. It has a constant-lnar-constant bhavor,.. t s constant byond th spcfd bound (-a to a).lk th OLS t assgns qual wghts to all obsrvatons wthn ts bound, whch surly wll rsult n ts hgh ffcncy but dstant outlrs stll hav a maxmum nflunc (n th form of constant a), whch lad to th ffcncy losss of about - prcnt n typcal cass wth outlrs (Hampl t al 986). To cop wth ths problm rdscndng M-stmators wr ntroducd. 4. Rdscndng M-Estmators Rdscndng M-stmators ar vry popular -typ M-Estmator whch has functons that ar nondcrasng nar th orgn, but dcrasng toward far from th orgn. Thr functons can b chosn to rdscnd smoothly to zro, so that thy usually satsfy x for all x wth X k,whr k s rfrrd to as th mnmum rjctd pont. Whn choosng a rdscndng functons w must tak car that t dos not dscnd too stply, whch may hav a vry bad nflunc on th dnomnator n th xprsson for th asymptotc varanc df ' df (9) whr F s th mxtur modl dstrbuton. Ths ffct s partcularly harmful whn a larg ngatv valus of ' ( x) combns wth a larg postv valus and Radha, ). Hubr s nflunc functon s as follows; ( x), and thr s a clustr of outlrs nar x (Muthukrshnan H k x k x x x k k x k () In Hubr s functon, th wghtng of rrors n M-stmaton ar calculatd by th followng functon; w H x x k k / x x k () Andrw s nfluncand wght functons: A x sn x / k x k. () x k w A x sn x/ k x/ k x x k k (3) Tuky s bwght M-stmator hav functons for any postv k, whch dfnd by Volum 5, Issu avalabl at 4

5 T x Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: x x x k k. (4) x k x x k wt x k x k (5) Ramsay s nflunc and wght functonsar as follows; kx maxmum at x x k R (6) R w x kx (7) Fgur ndcats a comparson M-stmator wght functons and th man. Fgur. M-Estmator Wght Functons Compard to th Man For rgrsson analyss, som of th rdcndng M-stmators can attan th maxmum brakdown pont. Morovr, som of thm ar th solutons of th problm of maxmzng th ffcncy undr boundd nflunc functon whn th rgrsson coffcnt and th scal paramtr ar stmatd smultanously. Hnc rdcndng M-stmators satsfy svral outlr robustnss proprts (Muthukrshnan and Radha, ). 5. Th Smulaton Study Th smulaton study valuats lnar multpl rgrsson analyss wth two ndpndnt varabls as shown n (8). WLS and WLSR mthods ar compard accordng to th MAE of and th MAE of y : y x x,(8) Volum 5, Issu avalabl at 43

6 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: whr y s th dpndnt varabl, x and x ar ndpndnt varabls, s th rror,and rgrsson paramtrs.for OLS w hav ^ ^ ^ ^ lsr, lsr, lsr, lsr. ^ ^ ^ ^ ols, ols, ols, ols ; and also, LSR gvs ar th tru In th smulaton procss, th ndpndnt varabls x and x ar randomly gnratd from a normal dstrbuton wth and ; rgrsson modl bcoms as follows:, and ar qual to, so y x x. (9). Thus, th Fnally, rrors ar randomly gnratd as Gaussan wht nos wth varanc. Thrfor, th dpndnt varabl has a normal dstrbuton wth man and varanc + Th smulatons wr prformd by R, usng dffrnt sampl szs and rror varancs.durng calculaton of m stmators, OLS and LSR mthods wr usd to ft ntal rgrsson modl; ntal rsduals wr found, and thy wr scald by MAD; a chosn wght functon was appld to obtan prlmnary wghts.th prlmnary wghts wr usd n tratvly rwghtd last squars and tratvly rwghtd last squars rato mthods to obtan rgrsson paramtrs; scondary rsduals wr found durng th frst traton. In th scond and othr tratons, th rsduals wr scald by Hubr proposal untll th bst modl was found. Th followng crtra wr usd to obtan th fnal stmats;. ^ ( q) ^ ( q) ^ ( q) () whr q rfrs to th numbr of tratons; ndcats a vry small postv numbr. In ths study, took th valu of.. Volum 5, Issu avalabl at 44

7 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl. Comparson of WLS and WLSR for sampl sz 3 wth non-outlr n = 3 ma ma y ols lsr r r r r ols lsr r r r r ols lsr r r r r ols lsr r r r r Tabl ndcats th comparson of WLS and WLSR for sampl sz 3 wth non-outlr and dffrnt rror varancs. Accordng to ma and ma y, w can say that WLS s a lttl mor succssful than WLSR n cas of non-outlr and ncrasng rror varanc. Volum 5, Issu avalabl at 45

8 Outlrs Tabl ndcats th comparson of WLS and WLSR for sampl sz 3 wth Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl. Comparson of WLS and WLSR for sampl sz 3 wth = n = 3 ma ma y % ols lsr r r r r % ols lsr r r r r % ols lsr r r r r =. Accordng to ma and ma y, w can say that WLS s a lttl bt succssful than WLSR n cas of non-outlr. Excpt for Andrw s functon, w can also say that WLSR gvs bttr rsults than WLS n cas of ncrasng th outlr. Volum 5, Issu avalabl at 46

9 Outlrs Tabl 3 ndcats th comparson of WLS and WLSR for sampl sz 3 wth Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl 3. Comparson of WLS and WLSR for sampl sz 3 wth = 9 n = 3 ma ma y % ols lsr r r r r % ols lsr r r r r % ols lsr r r r r =9. Accordng to ma and ma y, w can say agan that WLS s a lttl bt succssful than WLSR n cas of non-outlr. Excpt for Andrw s functon, w can also say that WLSR gvs bttr rsults than WLS n cas of ncrasng th outlr. Volum 5, Issu avalabl at 47

10 Outlrs Tabl 4 ndcats th comparson of WLS and WLSR for sampl sz 3 wth Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl 4. Comparson of WLS and WLSR for sampl sz 3 wth = 5 n = 3 ma ma y % ols lsr r r r r % ols lsr r r r r % ols lsr r r r r = 5. Accordng to ma and ma y, w can say agan that WLS s mor succssful than WLSR n cas of non-outlr. Excpt for Andrw s functon, w can also say that WLSR gvs bttr rsults than WLS n cas of ncrasng th outlr. Volum 5, Issu avalabl at 48

11 Tabl 5 ndcats th comparson of WLS and WLSR for sampl sz 3 wth Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl 5. Comparson of WLS and WLSR for sampl sz 3 wth = Outlrs n = 3 ma ma y % ols lsr r r r r % ols lsr r r r r % ols lsr r r r r =. Accordng to ma and ma y, w can say agan that WLS s mor succssful than WLSR n cas of non-outlr. Excpt for Andrw s functon, w can also say that WLSR gvs bttr rsults than WLS n cas of ncrasng th outlr.accordng to th frst fv tabls, WLS has bttr prformanc than WLSR ncas of non-outlr.also, WLSR gvs bttr rsults than WLS n cas of ncrasng th outlr and ncrasng th varanc. Volum 5, Issu avalabl at 49

12 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: ma ma y ma ma y ma ma y ma ma y ma ma y ols lsr r r r r ols lsr r r r r ols lsr r r r r ols lsr r r r r Tabl 6 ndcats th comparson of WLS and WLSR wth non-outlr for dffrnt sampl szs and rror varancs. Accordng to ma and ma y, w can say that WLS mthod outprforms than WLSR n cas of ncrasng sampl sz and varanc. Tabl 6. Comparson of WLS and WLSR wth non-outlr n Volum 5, Issu avalabl at 4

13 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl 7. Comparson of WLS and WLSR wth %*n outlrs, whch ar qual to 5 n ma ma y ma ma y ma ma y ma ma y ma ma y ols lsr r r r r ols lsr r r r r ols lsr r r r r ols lsr r r r r Tabl 7 ndcats th comparson of WLS and WLSR wth %*n outlrs for dffrnt sampl szs and rror varancs. Accordng to ma and ma y, xcpt for Andrw s and Tuky s functons w can say that WLSR gvs bttr rsults than WLS n cas of ncrasng th sampl sz and th rror varanc. Volum 5, Issu avalabl at 4

14 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl 8. Comparson of WLS and WLSR wth %*n outlrs, whch ar qual to 5 n ma ma y ma ma y ma ma y ma ma y ma ma y ols lsr r r r r ols lsr r r r r ols lsr r r r r ols lsr r r r r Tabl 8 ndcats th comparson of WLS and WLSR wth %*n outlrs for dffrnt sampl szs and rror varancs. Accordng to ma and ma y, xcpt for Andrw s functon w can say that WLSR gvs bttr rsults than WLS n cas of ncrasng th sampl sz and th rror varanc. Volum 5, Issu avalabl at 4

15 % Outlrs % Outlrs Non-Outlr Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: Tabl 9. Comparson of th succss of WLS and WLSR Mthods, xcpt for Andrw s functon n ma ma y ma ma y ma ma y ma ma y ma ma y 9 5 r r r r r r r r r r 9 r r r r r r r r r r 5 r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r 9 r r r r r r r r r r 5 r r r r r r r r r r r r r r r r r r r r Tabl 9 ndcats th comparson of th succss of WLS and WLSR Mthods wth dffrnt rror varancs and outlrs rats. W can say that WLS Mthod outprform than WLSR Mthod n cas of non-outlr. W can also say that WLSR Mthod gvs bttr rsults than WLS Mthod n cas of ncrasng outlr ratos, and th rror varanc. 6. Conclusons and Futur Work In ths study, t s shown whch mthod (WLS and WLSR) gvs bttr rsults n M-stmaton accordng to th man absolut rrors (MAE) of th stmatd rgrsson paramtrs and dpndnt valu va a smulaton study usng dffrnt sampl szs and rror varancs. It was studd on Hubr, Tuky, Andrw and Ramsay s wghtng functons n ths papr. Basd on th smulaton rsults, w can say that WLSR gvs bttr stmats than WLS n cas of th prsnc of outlrs and ncrasd rror varanc apart from Andrw s wghtng functon. For futur work, othr wghtng functons n th ltratur can b xamnd for whch mthod gvs bttr rsults. Acknowldgmnts I would lk to thank JForc Informaton Tchnologs Inc. Istanbul, Turky for supportng ths study. I would also lk to Dr. Oguz Akblgc for sharng hs knowldg wth m. Rfrncs [] Akblgc O., Aknc E. D., 9. A novl rgrsson approach: Last squars rato. Communcatons n Statstcs - Thory and Mthods, 38:9, , [] Andrsn R., 7. Modrn Mthods for robust rgrsson. Sag Publcatons. [3] Andrws D. F., 974. A Robust mthod for multpl lnar rgrsson. Tchnomtrcs, Vol.6, [4] Andrws D. F., Bckl P. J., Hampl F. R., Hubr P. J., Rogrs W. H., and Tuky J. W., 97. Robust stmats of locaton. (Prncton Unv. Prss, Prncton). [5] Banas M., Lgas M., 4. Emprcal tsts of prformanc of som M stmators. Godsy and Cartography. Vol. 63, No, 4, pp [6] Chattrj S., Machlr M., 997. Robust rgrsson:a wghtd last squars approach. Communcatons n Statstcs - Thory and Mthods, 6(6), Volum 5, Issu avalabl at 43

16 Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: [7] Chang X.-W., Guo Y., 5. Hubr s m-stmaton n rlatv GPS postonng: computatonal aspcts. Journal of Godsy, Volum 79, Issu 6-7, pp [8] Drappr N. R., Smth H., 998. Appld rgrsson analyss. Nw York: Wly. [9] Duchnowsk, R.. Snstvty of robust stmators appld n stratgy for tstng stablty of rfrnc ponts. EIF approach. Godsy and Cartography, 6(), 3-34 [] Hoagln D. C, Mostllr F., and Tuky J. W., 983. Undrstandng robust and xploratory data analyss. John Wly and Sons, Nw York. [] Hubr P. J., 964. Robust stmaton of a locaton paramtr. Th Annals of Mathmatcal Statstcs, 35(), 73-. [] Kollr M., Stahl W. A.,. Sharpnng wald-typ nfrnc n robust rgrsson for small sampls. Computatonal Statstcs & Data Analyss 55(8), [3] Muthukrshnan R., Radha M.,. M-stmators n rgrsson modls. Journal of Mathmatcs Rsarch. Vol., No. 4, pp [4] Roussuw P. J., Lroy A. M., 3. Robust rgrsson and outlr dtcton. Wly-Intrscnc; frst dton. Volum 5, Issu avalabl at 44