Stochastic Restricted Maximum Likelihood Estimator in Logistic Regression Model

Transcription

1 Open Journal of Statstcs, 05, 5, Publshed Onlne December 05 n ScRes. Stochastc Restrcted Maxmum Lkelhood Estmator n Logstc Regresson Model Varathan Nagarajah,, Pushpakanthe Wjekoon 3 Postgraduate Insttute of Scence, Unversty of Peradenya, Peradenya, Sr Lanka Department of Mathematcs and Statstcs, Unversty of Jaffna, Jaffna, Sr Lanka 3 Department of Statstcs and Computer Scence, Unversty of Peradenya, Peradenya, Sr Lanka Receved November 05; accepted 7 December 05; publshed 30 December 05 Copyrght 05 by authors and Scentfc Research Publshng Inc. Ths work s lcensed under the Creatve Commons Attrbuton Internatonal Lcense CC BY. Abstract In the presence of multcollnearty n logstc regresson, the varance of the Maxmum Lkelhood Estmator becomes nflated. Şray et al. 05 [] proposed a restrcted Lu estmator n logstc regresson model wth exact lnear restrctons. However, there are some stuatons, where the lnear restrctons are stochastc. In ths paper, we propose a Stochastc Restrcted Maxmum Lkelhood Estmator SR for the logstc regresson model wth stochastc lnear restrctons to overcome ths ssue. Moreover, a Monte Carlo smulaton s conducted for comparng the performances of the, Restrcted Maxmum Lkelhood Estmator R, Rdge Type Logstc EstmatorLRE, Lu Type Logstc EstmatorLLE, and SR for the logstc regresson model by usng Scalar Mean Squared Error SMSE. Keywords Logstc Regresson, Multcollnearty, Stochastc Restrcted Maxmum Lkelhood Estmator, Scalar Mean Squared Error. Introducton In many felds of study such as medcne and epdemology, t s very mportant to predct a bnary response varable, or to compute the probablty of occurrence of an event, n terms of the values of a set of explanatory varables related to t. For example, the probablty of sufferng a heart attack s computed n terms of the levels of a set of rsk factors such as cholesterol and blood pressure. The logstc regresson model serves admrably ths purpose and s the most used for these cases. The general form of logstc regresson model s How to cte ths paper: Nagarajah, V. and Wjekoon, P. 05 Stochastc Restrcted Maxmum Lkelhood Estmator n Logstc Regresson Model. Open Journal of Statstcs, 5,

2 V. Nagarajah, P. Wjekoon y = π + ε, =,, n whch follows Bernoull dstrbuton wth parameter π as exp x β π =, + exp x β row of X, whch s an n p ε s ndependent wth mean zero and varance π π where x s the th + data matrx wth p explanatory varables and β s a p + vector of coeffcents, of the response y. The maxmum lkelhood method s the most common estmaton technque to estmate the parameter β, and the Maxmum Lkelhood Estmator of β can be obtaned as follows: where C = X WX ; Z s the column vector wth, C X WZ β = 3 th element equals logt π y + π π π W = dag π π, whch s an unbased estmate of β. The covarance matrx of β s Cov β { } X W X and =. 4 As many authors have stated Hosmer and Lemeshow 989 [] and Ryan 997 [3], among others, the logstc regresson model becomes unstable when there exsts strong dependence among explanatory varables mult-collnearty. For example, we suppose that the probablty of a person survvng 0 or more extra years s modelled usng three predctors Sex, Dastolc blood pressure and Body mass ndex. Snce the response whether the person survvng 0 or more extra years s bnary, the logstc regresson model s approprate for ths problem. However, t s understood that the predctors Sex, Dastolc blood pressure and Body mass ndex may have some nter-relatonshp wthn each person. In ths case, the estmaton of the model parameters becomes naccurate because of the need to nvert near-sngular nformaton matrces. Consequently, the nterpretaton of the relatonshp between the response and each explanatory varable n terms of odds rato may be erroneous. As a result, the estmates have large varances and large confdence ntervals, whch produce neffcent estmates. To overcome the problem of mult-collnearty n the logstc regresson, many estmators are proposed alternatves to the. The most popular way to deal wth ths problem s called the Rdge Logstc Regresson RLR, whch s frst proposed by Schaffer et al. 984 [4]. Later Prncpal Component Logstc Estmator PCLE by Agulera et al. 006 [5], the Modfed Logstc Rdge Regresson Estmator MLRE by Nja et al. 03 [6], Lu Estmator by Mansson et al. 0 [7], and Lu-type estmator by Inan and Erdogan 03 [8] n logstc regresson have been proposed. An alternatve technque to resolve the mult-collnearty problem s to consder parameter estmaton wth pror avalable lnear restrctons on the unknown parameters, whch may be exact or stochastc. That s, n some practcal stuatons there exst dfferent sets of pror nformaton from dfferent sources lke past experence or long assocaton of the expermenter wth the experment and smlar knd of experments conducted n the past. If the exact lnear restrctons are avalable n addton to logstc regresson model, many authors propose dfferent estmators for the respectve parameter β. Duffy and Santer 989 [9] ntroduce a Restrcted Maxmum Lkelhood Estmator R by ncorporatng the exact lnear restrcton on the unknown parameters. Recently Şray et al. 05 [] proposes a new estmator called Restrcted Lu Estmator RLE by replacng by R n the logstc Lu estmator. In ths paper we propose a new estmator whch s called as the Stochastc Restrcted Maxmum Lkelhood Estmator SR when the lnear stochastc restrctons are avalable n addton to the logstc regresson model. The rest of the paper s organzed as follows. The proposed estmator and ts asymptotc propertes are gven n Secton. In Secton 3, the mean square error matrx and the scalar mean square error for ths new estmator are obtaned. Secton 4 descrbes some mportant exstng estmators for the logstc regresson models. Performance of the proposed estmator wth respect to Scalar Mean Squared Error SMSE s compared wth some exstng estmators by performng a Monte Carlo smulaton study n Secton 5. The concluson of the study s presented n Secton

3 V. Nagarajah, P. Wjekoon. The Proposed Estmator and ts Asymptotc Propertes Frst consder the multple lnear regresson model y = Xβ + ε, ε ~ N 0, σ I, 5 where y s an n observable random vector, X s an n p known desgn matrx of rank p, β s a p vector of unknown parameters and ε s an n vector of dsturbances. The Ordnary Least Square Estmator OLSE of β s gven by, OLSE S Xy β = 6 where S= XX. In addton to sample model 5, consder the followng lnear stochastc restrcton on the parameter space β ; r = Rβ + ν; E ν = 0, Cov ν =Ω> 0 7 where r s an m stochastc known vector, R s a m p of full rank m p wth known elements and ν s an m random vector of dsturbances wth mean 0 and dsperson matrx Ω, and Ω s assumed to be known m m postve defnte matrx. Further t s assumed that ν s stochastcally ndependent of ε,.e. E εν = 0. The Restrcted Ordnary Least Square Estmator ROLSE due to exact pror restrcton.e. ν = 0 n 7 s gven by β = β + S R RS R r R β 8 ROLSE OLSE OLSE Thel and Goldberger 96 [0] proposed the mxed regresson estmator ME for the regresson model. wth the stochastc restrcted pror nformaton 7 β = β + S R Ω+ RS R r R β 9 ME OLSE OLSE Suppose that the followng lnear pror nformaton s gven n addton to the general logstc regresson model h = H β + υ; E υ = 0, Cov υ = Ψ 0 where h s an q stochastc known vector, H s a q p+ of full rank q p+ known elements and υ s an q random vector of dsturbances wth mean 0 and dsperson matrx Ψ, and Ψ s assumed to be known q q postve defnte matrx. Further, t s assumed that υ s stochastcally ndependent of * =,,, n,.e. * ε ε ε ε E ευ = 0. Duffy and Santner 989 [9] proposed the Restrcted Maxmum Lkelhood Estmator R for the logstc regresson model wth the exact pror restrcton.e. υ = 0 n 0 β = β + C H HC H h H β R Followng R n and the Mxed Estmator ME n 9 n the Lnear Regresson Model, we propose a new estmator whch s named as the Stochastc Restrcted Maxmum Lkelhood Estmator SR when the lnear stochastc restrcton 0 s avalable n addton to the logstc regresson model. Asymptotc Propertes of SR: The β s asymptotcally unbased. SR β = β + C H Ψ+ HC H h H β SR SR E β The asymtotc covarance matrx of SR equals = β 3 839

4 V. Nagarajah, P. Wjekoon β SR = Ψ+ 4 Var C C H HC H HC 3. Mean Square Error Matrx Comparsons To compare dfferent estmators wth respect to the same parameter vector β n the regresson model, one can use the well known Mean Square Error MSE Matrx MSE and/or Scalar Mean Square Error SMSE crtera. where β β = β β β β = β + β β MSE, E D B B D β s the dsperson matrx, and B E β = β β denotes the bas vector. The Scalar Mean Square Error SMSE of the estmator β can be defned as β β = trace MSE β β SMSE,, 6 For two gven estmators β and β, the estmator β s sad to be superor to β under the MSE crteron f and only f The MSE and SMSE of the proposed estmator SR s 5 M β, β = MSE β, β MSE β, β 0. 7 βsr = βsr + βsr βsr = C C H Ψ+ HC H HC MSE D B B β SR = Ψ+ SMSE trace C C H HC H HC β β SR β βsr Gan n Effcency = MSE MSE = D D = C H Ψ+ HC H HC Note that the dfference gven n 0 s non-negatve defnte. Thus by the MSE crtera t follows that β has smaller Mean square error than β. SR 4. Some Exstng Logstc Estmators To examne the performance of the proposed estmator SR over some exstng estmators, the followng estmators are consdered. Logstc Rdge Estmator Schaefer et al. 984 [4] proposed a rdge estmator for the logstc regresson model. β LRE = X WX + ki X WX β = C + ki C β = Z β k where 0 k > s the rdge parameter and The asymptotc MSE and SMSE of MSE where δ = Zk β β. Z = C + ki C. k LRE β, βlre = E βlre β βlre β D βlre B βlre B = + βlre = Z C Z + Z β β Z β β = C + ki C C + ki + δ δ k k k k 840

5 V. Nagarajah, P. Wjekoon SMSE β trace LRE LRE = E β β βlre β β β ZZ k k β β Zkβ β Zkβ β = trace + = trace C Z kzk + k β C + ki Logstc Lu Estmator Followng Lu 993 [], Urgan and Tez 008 [], Mansson et al. 0 [7] examned the Lu Estmator for logstc regresson model, whch s defned as where 0 d β LLE C I C di = + + = Z β d < < s a parameter and Z C I C di The asymptotc MSE and SMSE of where δ = Zd β β. SMSE MSE = + +. β, d LLE βlle = E βlle β βlle β β β LLE βlle βlle = D + B B β β β β β = ZC Z + Z Z d d d d { } { } C I C di C C I C di = δδ C I C di I dc C I = δδ βlle = trace E βlle β βlle β β β Z dzd β β Zdβ β Zdβ β = trace + = trace C Z dz d + k β C + ki β 3 Restrcted As we mentoned n Secton, Duffy and Santner 989 [9] proposed the Restrcted Maxmum Lkelhood Estmator R for the logstc regresson model wth the exact pror restrcton.e. υ = 0 n 0. The asymptotc MSE and SMSE of R β = β + C H HC H h H β 7 R β, SMSE MSE A = D β = C C H HC H HC where R and δ3 = Bas βr = C H HC H h H β Mean Squared Error Comparsons R β A δδ 3 3 = + 8 βr trace A δδ 3 3 =

6 V. Nagarajah, P. Wjekoon SR versus LRE β LRE MSE βsr { D β } { LRE D βsr B βlre B βlre B βsr B βsr } { C ki C C ki } { C C H HC H HC } δδ { C ki C C ki } C H H δδ { C ki C C ki δδ } C H H MSE = + = + + Ψ+ + = Ψ + = Ψ M N = where = and M C ki C C ki δδ C ki C C ki N = C+ H Ψ H. One can obvously say that + + and N are postve defnte and δδ s non-negatve defnte matrces. Further by Theorem see Appendx, t s clear that M s postve defnte matrx. By Lemma see Appendx, f λmax NM <, where λmax NM s the largest egen value of NM then M N s a postve defnte matrx. Based on the above arguments, the followng theorem can be stated. λ NM <. Theorem 4.. The estmator SR s superor to LRE f and only f SR Versus LLE MSE β MSE max LLE βsr { D βlle D βsr } { B βlle B βlle B βsr B βsr } { } { } C I C di I dc C I C H H δδ { C I C di I dc } C I δδ C H H = + M N { } C I C di I dc C I C C H HC H HC δδ = Ψ+ + = Ψ + = Ψ = { } where M C I C di I dc = C + I + δδ and N C H H say that C I C di I dc C I 30 3 = + Ψ. One can obvously and N are postve defnte and δδ s non-negatve def- nte matrces. Further by Theorem see Appendx, t s clear that M s postve defnte matrx. By Lemma see Appendx, f λmax NM <, where λmax NM s the the largest egen value of NM then M N s a postve defnte matrx. Based on the above arguments, the followng theorem can be stated. Theorem 4.. The estmator SR s superor to LLE f and only f λmax NM <. SR versus R βr MSE βsr { D βr D βsr } { B βr B βr B βsr B βsr } C C H HC H HC C C H HC H HC MSE = + { } { } δδ 3 3 { } C H HC H HC C H HC H HC δδ 3 3 { C H HC H HC δδ 3 3} { C H HC H } HC = M 3 N3 = Ψ+ + = Ψ+ + = Ψ

7 { } { } where M 3 = C H Ψ+ HC H HC + δδ 3 3 and 3 say that V. Nagarajah, P. Wjekoon N = C H HC H HC. One can obvously C H Ψ+ HC H HC and N 3 are postve defnte and δδ 3 3 s non-negatve defnte matrces. Further by Theorem see Appendx, t s clear that M 3 s postve defnte matrx. By Lemma see Appendx, f λmax NM 3 3 <, where λmax NM 3 3 M 3 3 s the the largest egen value of NM 3 3 N s a postve defnte matrx. Based on the above arguments, the followng theorem can be stated. Theorem 4.3. The estmator SR s superor to R f and only f λ NM <. max 3 3 Based on the above results one can say that the new estmator SR s superor to the other estmators wth respect to the mean squared error matrx sense under certan condtons. To check the superorty of the estmators numercally, we then consder a smulaton study n the next secton. 5. A Smulaton Study A Monte Carlo smulaton s done to llustrate the performance of the new estmator SR over the, R, LRE, and LLE by means of Scalar Mean Square Error SMSE. Followng McDonald and Galarneau 975 [3] the data are generated as follows: then xj = ρ zj + ρ z, p+, =,,, n, j =,,, p 33 where z j are pseudo- random numbers from standardzed normal dstrbuton and ρ represents the correlaton between any two explanatory varables. Four explanatory varables are generated usng 33. We consdered four dfferent values of ρ correspondng to 0.70, 0.80, 0.90 and Further four dfferent values of n correspondng to 0, 40, 50, and 00 are consdered. The dependent varable y n s obtaned from the Berexp x β noull π dstrbuton where π =. The parameter values of β, β,, βp are chosen so that + exp x β p β j = j and β = β = = βp. Moreover, for the restrcton, we choose H = 0 0, h = and Ψ= Further for the rdge parameter k and the Lu parameter d, some selected values are chosen so that 0 k and 0 d. The experment s replcated 3000 tmes by generatng new pseudo-random numbers and the estmated SMSE s obtaned as { } tr E * SMSE β = Mean tr MSE β, β = Mean β β β β Mean E tr β β β β Mean E tr β β = = β β 3000 Mean β β β β = = β β β β 3000 nsm = The smulaton results are lsted n Tables A-A6 Appendx 3 and also dsplayed n Fgures A-A4 Appendx. From Fgures A-A4, t can be notced that n general ncrease n degree of correlaton between two explanatory varables ρ nflates the estmated SMSE of all the estmators and ncrease n sample sze n declnes the estmated SMSE of all the estmators. Further, the new estmator SR has smaller SMSE compared to wth respect to all the values of ρ and n. However, when n = 0 and 0.3 kd, 0.5,

8 V. Nagarajah, P. Wjekoon SR performs better compared to the estmators LRE, and LLE. From Table A7 Appendx 3, t s clear that when k and d are small LLE s better than other estmators n the MSE sense, and LRE s better when k and d are large. For moderate k and d values the proposed estmator s good, but ths wll change wth the n and ρ values. Therefore we then analyse the estmators LRE, LLE and SR further by usng dfferent k and d values and the results are summarzed n Table A8 and Table A9 Appendx 3. Accordng to these results t s clear that SR s even superor to LRE and LLE for certan values of k and d. 6. Concludng Remarks In ths research, we ntroduced the Stochastc Restrcted Maxmum Lkelhood Estmator SR for logstc regresson model when the lnear stochastc restrcton was avalable. The performances of the SR over, LRE, R, and LLE n logstc regresson model were nvestgated by performng a Monte Carlo smulaton study. The research had been done by consderng dfferent degree of correlatons, dfferent numbers of observatons and dfferent values of parameters k, d. It was noted that the SMSE of the was nflated when the multcollnearty was presented and t was severe partcularly for small samples. The smulaton results showed that the proposed estmator SR had smaller SMSE than the estmator wth respect to all the values of n and ρ. Further t was noted that the proposed estmator SR was superor over the estmators LLE and LRE for some k and d values related to dfferent ρ and n. Acknowledgements We thank the edtor and the referee for ther comments and suggestons, and the Postgraduate Insttute of Scence, Unversty of Peradenya, Sr Lanka for provdng necessary facltes to complete ths research. References [] Şray, G.U., Toker, S. and, Kaçranlar, S. 05 On the Restrcted Lu Estmator n Logstc Regresson Model. Communcatons n Statstcs Smulaton and Computaton, 44, [] Hosmer, D.W. and Lemeshow, S. 989 Appled Logstc Regresson. Wley, New York. [3] Ryan, T.P. 997 Modern Regresson Methods. Wley, New York. [4] Schaefer, R.L., Ro, L.D. and Wolfe, R.A. 984 A Rdge Logstc Estmator. Communcatons n Statstcs Theory and Methods, 3, [5] Agulera, A.M., Escabas, M. and Valderrama, M.J. 006 Usng Prncpal Components for Estmatng Logstc Regresson wth Hgh-Dmensonal Multcollnear Data. Computatonal Statstcs & Data Analyss, 50, [6] Nja, M.E., Ogoke, U.P. and Nduka, E.C. 03 The Logstc Regresson Model wth a Modfed Weght Functon. Journal of Statstcal and Econometrc Method,, 6-7. [7] Mansson, G., Kbra, B.M.G. and Shukur, G. 0 On Lu Estmators for the Logt Regresson Model. The Royal Insttute of Techonology, Centre of Excellence for Scence and Innovaton Studes CESIS, Paper No [8] Inan, D. and Erdogan, B.E. 03 Lu-Type Logstc Estmator. Communcatons n Statstcs Smulaton and Computaton, 4, [9] Duffy, D.E. and Santner, T.J. 989 On the Small Sample Prospertes of Norm-Restrcted Maxmum Lkelhood Estmators for Logstc Regresson Models. Communcatons n Statstcs Theory and Methods, 8, [0] Thel, H. and Goldberger, A.S. 96 On Pure and Mxed Estmaton n ECONOMICS. Internatonal Economc Revew,, [] Lu, K. 993 A New Class of Based Estmate n Lnear Regresson. Communcatons n Statstcs Theory and Methods,, [] Urgan, N.N. and Tez, M. 008 Lu Estmator n Logstc Regresson When the Data Are Collnear. Internatonal Conference on Contnuous Optmzaton and Knowledge-Based Technologes, Lnthuana, Selected Papers, Vlnus, [3] McDonald, G.C. and Galarneau, D.I. 975 A Monte Carlo Evaluaton of Some Rdge-Type Estmators. Journal of the 844

9 V. Nagarajah, P. Wjekoon Amercan Statstcal Assocaton, 70, [4] Rao, C.R. and Toutenburg, H. 995 Lnear Models: Least Squares and Alternatves. nd Edton, Sprnger-Verlag, New York, Inc. [5] Rao, C.R., Toutenburg, H., Shalabh and Heumann, C. 008 Lnear Models and Generalzatons. Sprnger, Berln. 845

10 V. Nagarajah, P. Wjekoon Appendx Theorem. Let A: n n and B: n n such that A > 0 and B 0. Then A+ B 0. Rao and Toutenburg, 995 [4]. Lemma. Let the two n n matrces M > 0, N 0, then M > N f and only f λmax NM <. Rao et al., 008 [5]. Appendx Fgure A. Estmated SMSE values for, LRE, R, LLE and SR for n = 0. Fgure A. Estmated SMSE values for, LRE, R, LLE and SR for n =

11 V. Nagarajah, P. Wjekoon Fgure A3. Estmated SMSE values for, LRE, R, LLE and SR for n = 75. Fgure A4. Estmated SMSE values for, LRE, R, LLE and SR for n = 00. Appendx 3 Table A. The estmated MSE values for dfferent k, d when n = 0 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR

12 V. Nagarajah, P. Wjekoon Table A. The estmated MSE values for dfferent k, d when n = 0 and ρ = 0.80 k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A3. The estmated MSE values for dfferent k, d when n = 0 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A4. The estmated MSE values for dfferent k, d when n = 0 and ρ = k, d=0.0 k, d=0. k, d=0. k, d=0.3 k, d=0.4 k, d=0.5 k, d=0.6 k, d=0.7 k, d=0.8 k, d=0.9 k, d=0.99 k, d= LRE R LLE SR Table A5. The estmated MSE values for dfferent k, d when n = 50 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A6. The estmated MSE values for dfferent k, d when n = 50 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A7. The estmated MSE values for dfferent k, d when n = 50 and ρ = 0.90 k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR

13 V. Nagarajah, P. Wjekoon Table A8. The estmated MSE values for dfferent k, d when n = 50 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A9. The estmated MSE values for dfferent k, d when n = 75 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A0. The estmated MSE values for dfferent k, d when n = 75 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A. The estmated MSE values for dfferent k, d when n = 75 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A. The estmated MSE values for dfferent k, d when n = 75 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A3. The estmated MSE values for dfferent k, d when n = 00 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR

14 V. Nagarajah, P. Wjekoon Table A4. The estmated MSE values for dfferent k, d when n = 00 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A5. The estmated MSE values for dfferent k, d when n = 00 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A6. The estmated MSE values for dfferent k, d when n = 00 and ρ = k, d = 0.0 k, d = 0. k, d = 0. k, d = 0.3 k, d = 0.4 k, d = 0.5 k, d = 0.6 k, d = 0.7 k, d = 0.8 k, d = 0.9 k, d = 0.99 k, d = LRE R LLE SR Table A7. Summary of the Tables A-A6. Best Estmator n = 0 ρ = 0.7 ρ = 0.8 ρ = 0.9 ρ = 0.99 LLE,,,, 0. SR 0.3 kd, kd, kd, 0.4 kd=, 0. LRE 0.6 kd, kd, kd, kd,.0 n = 50 LLE, 0.,,, SR 0. kd, kd, kd, 0.6 kd=, 0.3 LRE 0.8 kd, kd, kd, kd,.0 n = 75 LLE, 0., 0.,, SR 0. kd, kd, kd, 0.6 kd=, 0.3 LRE 0.9 kd, kd, kd, kd,.0 n = 00 LLE, 0., 0., 0., SR 0. kd, kd, kd, 0.7 kd=, 0.3 LRE 0.9 kd, kd, kd, kd,.0 850

15 V. Nagarajah, P. Wjekoon Table A8. The best estmators and the correspondng k, d values when n = 0 and n = 50. ρ = 0.7 ρ = 0.8 ρ = 0.9 ρ = 0.99 n = 0 n = 50 LLE 0. k 0.6, 0. d 0. k 0.8, d = 0. LRE 0.9 k 0.99, 0. d 0.9 k, d = k 0.8 k 0.99, 0. d SR 0. k 0.5, 0.3 d 0. k 0.7, 0. d LLE 0. k 0.6, 0. d 0. k 0.8, 0. d LRE 0.8 k 0.99, 0. d 0.9 k 0.99, 0. d 0.6 k 0.8 k SR 0. k 0.5, 0.3 d 0. k 0.7, 0.3 d LLE 0. k 0.5, 0. d 0. k 0.6, 0. d LRE 0.8 k 0.99, 0. d 0. k 0.6, 0. d 0.5 k 0.7 k SR 0. k 0.4, 0.3 d 0. k 0.6, 0.3 d LLE 0. k, d = k 0.5, d = 0. LRE 0.3 k, d = k, d = k 0.99, 0. d 0.4 k 0.99, 0. d SR 0. k 0., 0. d 0. k 0.3, 0. d Table A9. The best estmators and the correspondng k, d values when n = 75 and n = 00. ρ = 0.7 ρ = 0.8 ρ = 0.9 ρ = 0.99 n = 75 n = 00 LLE 0. k 0.8, d = k 0.8, d = 0. LRE 0.9 k, d = k 0.99, 0. d 0.8 k 0.99, 0. d SR 0. k 0.8, 0. d 0. k 0.8, 0. d LLE 0. k 0.8, d = k 0.8, d = 0. LRE 0.9 k, d = k 0.99, 0. d 0.8 k 0.99, 0. d SR 0. k 0.7, 0. d 0. k 0.8, 0. d LLE 0. k 0.7, 0. d 0. k 0.8, d = 0. LRE 0.9 k 0.99, 0. d 0.9 k, d = k 0.8 k 0.99, 0. d SR 0. k 0.6, 0.3 d 0. k 0.7, 0. d LLE 0. k 0.4, 0. d 0. k 0.4, 0. d LRE 0.5 k 0.99, 0. d 0.7 k 0.99, 0. d 0.4 k 0.4 k SR 0. k 0.3, 0.3 d 0. k 0.3, 0.3 d 85