On the Restricted Almost Unbiased Ridge Estimator in Logistic Regression

Transcription

1 Open Journal of Statstcs, 06, 6, ISSN Onlne: ISSN Prnt: 6-78X On the Restrcted Almost Unbased Rdge Estmator n Logstc Regresson Nagarajah Varathan,, Pushpaanthe Wjeoon 3 Postgraduate Insttute of Scence, Unversty of Peradenya, Peradenya, Sr Lana Department of Mathematcs and Statstcs, Unversty of Jaffna, Jaffna, Sr Lana 3 Department of Statstcs and Computer Scence, Unversty of Peradenya, Peradenya, Sr Lana How to cte ths paper: Varathan, N. and Wjeoon, P. (06) On the Restrcted Almost Unbased Rdge Estmator n Logstc Regresson. Open Journal of Statstcs, 6, Receved: September 0, 06 Accepted: December 3, 06 Publshed: December 8, 06 Copyrght 06 by authors and Scentfc Research Publshng Inc. Ths wor s lcensed under the Creatve Commons Attrbuton Internatonal Lcense (CC BY 4.0). Open Access Abstract In ths artcle, the restrcted almost unbased rdge logstc estmator (RAURLE) s proposed to estmate the parameter n a logstc regresson model wth exact lnear restrctons when there exsts multcollnearty among explanatory varables. The performance of the proposed estmator over the maxmum lelhood estmator (), rdge logstc estmator (RLE), almost unbased rdge logstc estmator (AURLE), and restrcted maxmum lelhood estmator (R) wth respect to dfferent rdge parameters s nvestgated through a smulaton study n terms of scalar mean square error. Keywords Multcollnearty, Rdge Estmator, Almost Unbased Rdge Logstc Estmator, Lnear Restrctons, Scalar Mean Square Error. Introducton Multcollnearty nflates the varance of the maxmum lelhood estmator () n the logstc regresson. As a result, one may not obtan an effcent estmate for the parameter β n the logstc regresson model. To combat the multcollnearty n logstc regresson, several alternatve technques have been proposed n the lterature. One of the most famous technques s to consder sutable based estmators n place of Maxmum lelhood estmator. The based estmators proposed n the lterature, are the Rdge Logstc Estmator (RLE) (Schaefer et al., 984 []), Lu Logstc Estmator (LLE) (Lu, 993 [], Urgan and Tez, 008 [3], and Mansson et al., 0 [4]), Prncpal Component Logstc Estmator (PCLE) (Agulera et al., 006 [5]), Modfed Logstc Rdge Estmator (MLRE) (Nja et al., 03 [6]), Lu-type estmator (Inan and Erdogan, 03 DOI: 0.436/ojs December 8, 06

2 N. Varathan, P. Wjeoon [7]), and Almost Unbased Lu Logstc Estmator (AULLE) (Xnfeng, 05 [8]). Morever, Asar (05) [9], proposed some new methods to solve the multcollnearty n logstc regresson by ntroducng new methods of estmatng the shrnage parameter n Lu-type estmators. Only the sample nformaton was used n all the above estmaton procedures. An alternatve technque suggested to solve the multcollnearty problem s to consder parameter estmaton wth some lnear restrctons on the unnown parameters, whch are generally based on pror nformaton of the sample data, and further they may be n the exact or stochastc form. By ncorporatng lnear restrctons to the sample nformaton, dfferent types of based estmators were ntroduced n the lterature, and some researchers have ncorporated these estmators wth the logstc regresson estmator to mprove ts performance. In the presence of exact lnear restrctons n addton to sample logstc regresson model, Duffy and Santer (989) [0] ntroduced the restrcted maxmum lelhood estmator (R) by ncorporatng the restrcted least squares estmator based on exact lnear restrcton to the logstc regresson. Later, the Restrcted Logstc Rdge Estmator (Asar et al., 06 []), Restrcted Logstc Lu Estmator (RLLE) (Şray et al., 05 []), Modfed Restrcted Lu Estmator (Wu, 06 [3]), Restrcted two parameter Lu type estmator (Asar et al., 06 [4]) were ntroduced to the logstc regresson wth exact lnear restrctons. In the presence of stochastc lnear restrctons n addton to sample logstc regresson model, Nagarajah and Wjeoon (05) ntroduced the Stochastc Restrcted Maxmum Lelhood Estmator (SR). Followng Nagarajah and Wjeoon (05) [5], the Stochastc Restrcted Rdge Maxmum Lelhood Estmator (SRR) was proposed by Varathan and Wjeoon (06) [6] by ncorporatng Rdge Logstc Estmator (RLE) wth the SR. Wu and Asar (06) [7] has proposed a new based estmator called Almost Unbased Rdge Logstc Estmator (AURLE), and shown ts performance over the other avalable estmators. In ths artcle, we further mprove the logstc regresson estmator by combnng AURLE wth R, and name t as the Restrcted Almost Unbased Rdge Logstc Estmator (RAURLE). Further, the performance of RAURLE based on estmated rdge parameters usng dfferent methods gven n the lterature was consdered, and compared each of these cases wth, RLE, AURLE and R. The proceedng sectons of the artcle are organzed as follows. The model specfcaton and estmaton are dscussed n Secton. The proposed estmator and ts asymptotc propertes are gven n Secton 3. Secton 4 descrbes the exstng methods related to some rdge parameters. In Secton 5, the performance of the proposed estmator by consderng dfferent rdge parameters s compared wth respect to the scalar mean squared error (SMSE) wth, RLE, AURLE and R by performng a Monte Carlo smulaton study. Fnally, conclusons of the study are presented n Secton 6.. Model Specfcaton and Estmaton Consder the followng logstc regresson model y π + ε,,, n () 077

3 N. Varathan, P. Wjeoon whch follows Bernoull dstrbuton wth parameter exp π + exp ( x β ) ( x β ), π as where x s the th row of X, whch s an n ( p ) varables and β s a ( p + ) vector of coeffcents, zero and varance π( π) of the response () of β can be obtaned as follows:, () + data matrx wth p predctor ε are ndependent wth mean y. The maxmum lelhood estmator β C X WZ (3) where C X WX ; Z s the column vector wth th element equals y π logt ( π ) + and W dag π ( π) π π, whch s an unbased estmate of ( ) β. The covarance matrx of β s Cov ( β ) { } X WX. (4) In the presence of multcollnearty, Schaefer et al. (984) [] proposed to ncorporate the Logstc Rdge Estmator (LRE), n place of the n the logstc regresson model () ( ) ( ) LRE X WX I X WX C I C Z β + β + β β (5) where ( ) Z C + I C and s the rdge parameter, 0. The asymptotc propertes of LRE: E β LRE E Zβ Zβ Cov β LRE Cov Z β ZC Z ( C I ) C ( C I ) Z ( C I ) However the LRE s a based estmator whch produces nconsstent estmates for the parameter (Wu and Asar, 06 [7]). Consequently, the Almost Unbased Rdge Logstc Estmator (AURLE) was ntroduced by Wu and Asar (06) [7] and t s defned as where ( ) ( ) β β β F I X WX + I. AURLE I X WX + I F And the asymptotc propertes of AURLE: E β AURLE E F β F β Cov β AURLE Cov F β F C F As another remedal acton for multcollnearty, one may use the exact lnear restrctons n addton to the sample logstc regresson model (). The resultng est- mator s called as Restrcted estmator. (6) (7) (8) (9) (0) 078

4 N. Varathan, P. Wjeoon Suppose that the followng exact restrcton s gven n addton to the general logstc regresson model (). Hβ h () where H s a ( q ( p+ ) ) nown matrx and h s an ( ) q vector of nown con- stants. In the presence of the above restrcton () n addton to the logstc regresson model (), Duffy and Santner (989) [0] proposed the followng Restrcted Maxmum Lelhood Estmator (R). ( ) ( ) R β β C H HC H Hβ h () The asymptotc mean and varance of R β are ( ) ( ) E β R E β C H HC H Hβ h ( ) ( β ) β C H HC H H h (3) and Consequently the bas of ( ) ( β ) R ( ) Cov C C H HC H HC A say. (4) R β, 3. The Proposed Estmator ( βr ) ( ) ( β ) Bas C H HC H H h. (5) To mprove the performance of the estmators further, n ths secton, by combnng AURLE and R, we propose a new estmator whch s called as the Restrcted Almost Unbased Rdge Logstc Estmator (RAURLE) and defned as ( ) β RAURLE I X WX + I βr F β R (6) where F ( ) I X WX + I. Note that ths estmator s based on the rdge parameter, and ts performance s based on the choce of. The asymptotc propertes of RAURLE β are E β RAURLE E F β R F β C H ( HC H ) ( Hβ h), (7) and ( ) ( ) ( ) ( ) D β Cov β Cov F β F Cov β F F AF, (8) RAURLE RAURLE R R ( βraurle ) β RAURLE β β ( ) ( β ) β δ ( ) Bas E F C H HC H H h say.(9) Consequently, the mean square error can be obtaned as, ( ) ( ) ( ) ( ) MSE β D β + Bas β Bas β F AF + δδ (0) RAURLE RAURLE RAURLE RAURLE 079

5 N. Varathan, P. Wjeoon 4. Some Rdge Estmators Now we consder the exstng methods to obtan an estmated value for the rdge parameter, snce RAURLE depends on. Many researchers suggested varous methods of estmatng the rdge parameter n the rdge regresson approach and recently ths estmaton method s added to the logstc regresson. In ths research, we have consdered the followng exstng rdge parameter estmaton methods to compare the performance of the proposed estmator wth some exstng estmators n logstc regresson. ) Hoerl and Kennard (970) [8]; HK σ () α where α max s the maxmum element of γβ, γ s the egen vector of X WX. ) Hoerl et al. (975) [9]; HKB max where p s the number of predctor varables n the model (). 3) Lawless and Wang (976) [0]; LW p σ () β β p σ (3) β X WX β 4) Lndley and Smth (97) []; LS ( )( + ) ( n ) n p p p σ + β β (4) 5) Schaefer et al. (984) []; 5. Smulaton Study HK (5) α max It s dffcult to compare the mean square error of the estmators theoretcally, snce none of the estmators, RLE, AURLE, R and RAURLE are not always superor. So, we use Monte Carlo smulaton to examne the performance of the proposed estmator over the exstng estmators under dfferent levels of multcollnearty. Followng McDonald and Galarneau (975) [] and Kbra (003) [3], the explanatory varables are generated usng the followng equaton. ( ) xj ρ zj + ρ z, p+,,,, n, j,,, p (6) where z j are ndependent pseudo standard normal random numbers and ρ represents the correlaton between any two explanatory varables. The n observatons for the response varable are obtaned from the Bernoull ( π ) dstrbuton n (). Four explanatory varables are generated usng (6) and four dfferent values of ρ correspondng 080

6 N. Varathan, P. Wjeoon to 0.80, 0.90, 0.95 and 0.99 are consdered. Further for the sample sze n, three dfferent values 5, 60, and 00 are also consdered. The parameter values of β, β,, βp are p chosen so that β j j and β β βp, whch s common restrctons n many smulaton studes. Further for the rdge parameter, fve dfferent choces are used as defned n the Equatons ()-(5). The smulaton s repeated 000 tmes by generatng new pseudo-random numbers and the smulated SMSE values of the estmators are obtaned usng the followng equaton. 000 ( * ) ( SMSE β β ) ( r β βr β) (7) 000 where r β s any estmator consdered n the r th smulaton. The smulaton results are gven n Tables -3. It can be notced from the Tables -3 that the scalar mean square error of the proposed estmator RAURLE s smaller compared to, RLE, AURLE and R, wth respect to all the selected values of n, ρ, and, consdered n ths research. Further, the new estmator RAURLE has better performance when SRW s used. 6. Concludng Remars In ths paper, we proposed a restrcted almost unbased rdge logstc estmator (RAURLE) n logstc regresson wth exact lnear restrctons when the explanatory varables are hghly correlated. Through a Monte Carlo smulaton study, we examned r Table. The estmated SMSE values for dfferent, when n 5. ρ Estmator HK SRW HKB LW LS RLE AURLE R RAURLE RLE AURLE R RAURLE RLE AURLE R RAURLE RLE AURLE R RAURLE

7 N. Varathan, P. Wjeoon Table. The estmated SMSE values for dfferent, when n 60. ρ Estmator HK SRW HKB LW LS RLE AURLE R RAURLE RLE AURLE R RAURLE RLE AURLE R RAURLE RLE AURLE R RAURLE Table 3. The estmated SMSE values for dfferent, when n 00. ρ Estmator HK SRW HKB LW LS RLE AURLE R RAURLE RLE AURLE R RAURLE RLE AURLE R RAURLE RLE AURLE R RAURLE

8 N. Varathan, P. Wjeoon the performance of the proposed estmator over some exstng estmators, RLE, AURLE and R n terms of scalar mean square error. Also, fve dfferent choces of exstng rdge parameter estmates were used to compare the estmators. The results show that the newly proposed estmator outperforms all the other estmators consdered n ths study under the selected values of n, ρ, and by means of SMSE. Acnowledgements We than the Edtor and the referee for ther comments and suggestons, and the Postgraduate Insttute of Scence, Unversty of Peradenya, Sr Lana for provdng necessary facltes to complete ths research. References [] Schaefer, R.L., Ro, L.D. and Wolfe, R.A. (984) A Rdge Logstc Estmator. Communcatons n Statstcs - Theory and Methods, 3, [] Lu, K. (993) A New Class of Based Estmate n Lnear Regresson. Communcatons n Statstcs - Theory and Methods,, [3] Urgan, N.N. and Tez, M. (008) Lu Estmator n Logstc Regresson When the Data Are Collnear. Internatonal Conference. Contnuous Optmzaton and Knowledge-Based Technologes, [4] Mansson, G., Kbra, B.M.G. and Shuur, G. (0) On Lu Estmators for the Logt Regresson Model. The Royal Insttute of Techonology, Centre of Excellence for Scence and Innovaton Studes (CESIS), Sweden, Paper No. 59. [5] Agulera, A.M., Escabas, M. and Valderrama, M.J. (006) Usng Prncpal Components for Estmatng Logstc Regresson wth Hgh-Dmensonal Multcollnear Data. Computatonal Statstcs & Data Analyss, 50, [6] Nja, M.E., Ogoe, U.P. and Ndua, E.C. (03) The Logstc Regresson Model wth a Modfed Weght Functon. Journal of Statstcal and Econometrc Method,, 6-7. [7] Inan, D. and Erdogan, B.E. (03) Lu-Type Logstc Estmator. Communcatons n Statstcs - Smulaton and Computaton, 4, [8] Xnfeng, C. (05) On the Almost Unbased Rdge and Lu Estmator n the Logstc Regresson Model. Internatonal Conference on Socal Scence, Educaton Management and Sports Educaton, Atlants Press, Amsterdam, [9] Asar, Y. (05) Some New Methods to Solve Multcollnearty n Logstc Regresson. Communcatons n Statstcs - Smulaton and Computaton, Onlne. [0] Duffy, D.E. and Santner, T.J. (989) On the Small Sample Prospertes of Norm-Restrcted Maxmum Lelhood Estmators for Logstc Regresson Models. Communcatons n Statstcs - Theory and Methods, 8, [] Asar, Y., Arash, M. and Wu, J. (06) Restrcted Rdge Estmator n the Logstc Regresson Model. Communcatons n Statstcs - Smulaton and Computaton, Onlne. [] Şray, G.U., Toer, S. and Kaçranlar, S. (05) On the Restrcted Lu Estmator n Logstc Regresson Model. Communcatons n Statstcs - Smulaton and Computaton, 44, 7-083

9 N. Varathan, P. Wjeoon 3. [3] Wu, J. (06) Modfed Restrcted Lu Estmator n Logstc Regresson Model. Computatonal Statstcs, 3, [4] Asar, Y., Erşoğlu, M. and Arash, M. (06) Developng a Restrcted Two Parameter Lu-Type Estmator: A Comparson of Restrcted Estmators n the Bnary Logstc Regresson Model. Communcatons n Statstcs - Theory and Methods, Onlne. [5] Nagarajah, V. and Wjeoon, P. (05) Stochastc Restrcted Maxmum Lelhood Estmator n Logstc Regresson Model. Open Journal of Statstcs, 5, [6] Varathan, N. and Wjeoon, P. (06) Rdge Estmator n Logstc Regresson under Stochastc Lnear Restrcton. Brtsh Journal of Mathematcs & Computer Scence, 5,. [7] Wu, J. and Asar, Y. (06) On Almost Unbased Rdge Logstc Estmator for the Logstc Regresson Model. Hacettepe Journal of Mathematcs and Statstcs, 45, [8] Hoerl, E. and Kennard, R.W. (970) Rdge Regresson: Based Estmaton for Nonorthogonal Problems. Technometrcs,, [9] Hoerl, E., Kennard, R.W. and Baldwn, K.F. (975) Rdge Regresson: Some Smulatons. Communcatons n Statstcs, 4, [0] Lawless, J.F. and Wang, P. (976) A Smulaton Study of Rdge and Other Regresson Estmators. Communcatons n Statstcs - Theory and Methods, 4, [] Lndley, D.V. and Smth, A.F.M. (97) Bayes Estmate for the Lnear Model (wth Dscusson) Part. Journal of the Royal Statstcal Socety, Ser B, 34, -4. [] McDonald, G.C. and Galarneau, D.I. (975) A Monte Carlo Evaluaton of Some Rdge-Type Estmators. Journal of the Amercan Statstcal Assocaton, 70, [3] Kbra, B.M.G. (003) Performance of Some New Rdge Regresson Estmators. Communcatons n Statstcs - Theory and Methods, 3,

10 Submt or recommend next manuscrpt to SCIRP and we wll provde best servce for you: Acceptng pre-submsson nqures through Emal, Faceboo, LnedIn, Twtter, etc. A wde selecton of journals (nclusve of 9 subjects, more than 00 journals) Provdng 4-hour hgh-qualty servce User-frendly onlne submsson system Far and swft peer-revew system Effcent typesettng and proofreadng procedure Dsplay of the result of downloads and vsts, as well as the number of cted artcles Maxmum dssemnaton of your research wor Submt your manuscrpt at: Or contact ojs@scrp.org