Efficiency of the principal component Liu-type estimator in logistic

Transcription

1 Effcency of the pncpal component Lu-type estmato n logstc egesson model Jbo Wu and Yasn Asa 2 School of Mathematcs and Fnance, Chongqng Unvesty of Ats and Scences, Chongqng, Chna 2 Depatment of Mathematcs-Compute Scences, Necmettn Ebakan Unvesty, Konya, 42090, Tukey Jbo Wu:lnfen52@26.com Yasn Asa: yasa@konya.edu.t, yasnasa@hotmal.com

2 Effcency of the pncpal component Lu-type logstc estmato n logstc egesson model In ths pape we popose a pncpal component Lu-type logstc estmato by combnng the pncpal component logstc egesson estmato and Lu-type logstc estmato to ovecome the multcollneaty poblem. The supeoty of the new estmato ove some elated estmatos ae studed unde the asymptotc mean squaed eo matx. A Monte Calo smulaton expement s desgned to compae the pefomances of the estmatos usng mean squaed eo cteon. Fnally, a concluson secton s pesented. Keywods: Lu-type logstc estmato; Mean squaed eo matx; Maxmum lkelhood estmato Subject classfcaton codes: 62J07; 62J2. Intoducton Consde the followng bnay logstc egesson model x exp exp,,..., n x (.) denotes the th ow of X whch s an n p p q whee x x xq data matx wth q known covaate vectos, y shows the esponse vaable whch takes on the value ethe 0 o wth y ~ Benoull ( ), y s ae supposed to be ndependent of one anothe stands fo a p vecto of paametes. and 0 q Usually the maxmum lkelhood (ML) method s used to estmate. The coespondng log-lkelhood equaton of model (.) s gven by n log log (.2) L y y whee s the th element of the vecto,,2,..., n. 2

3 ML estmato can be obtaned by maxmzng the log-lkelhood equaton gven n (.2). Snce the equaton (.2) s non-lnea n, one should use an teatve algothm called teatvely e-weghted least squaes algothm (IRLS) as follows (Saleh and Kba, 203) : whee t s the estmated values of usng ˆt X V X X V y ˆ t ˆ t t t t ˆ (.3) t t t and V dag ˆ ˆ such that ˆ t s the th element of ˆt. Afte some algeba, Equaton (.3) can be wtten as follows: whee z z z n ˆML X VX X Vz (.2) wth x and z ( y )( / ). In lnea egesson analyss, multcollneaty has been egaded as a poblem n the estmaton. In dealng wth ths poblem, many ways have been ntoduced to deal wth ths poblem. One appoach s to study the based estmato such as dge estmato (Hoel and Kennad, 970), Lu estmato (Lu, 993), Lu-type estmato (Huang et al., 2009). Altenatvely, many authos such as Xu and Yang (20) and L and Yang (20), have studed the estmaton of lnea models wth addtonal estctons. As n lnea egesson, estmaton n logstc egesson s also senstve to multcollneaty. When thee s multcollneaty, columns of the matx X VX become close to be dependent. It mples that some of the egenvalues of X VX become close to zeo. Thus, mean squaed eo value of MLE s nflated so that one cannot obtan stable estmatons. Thus many authos have studed how to educe the multcollneaty, such as Lesaffe and Max (993) dscussed the multcollneaty n logstc egesson, Schaefe et al. (984) poposed the dge logstc (RL) estmato, Agulea et al. (2006) poposed the pncpal component logstc egesson (PCLR) estmato, Masson et al. (202), 3

4 ntoduced the Lu logstc (LL) estmato, by combnng the pncpal component logstc egesson estmato and dge logstc estmato to deal wth multcollneaty. Moeove, Inan and Edoğan (203) poposed Lu-type logstc estmato (LTL) and Asa (207) studed some popetes of LTL. In ths study, by combnng the pncpal component logstc egesson estmato and the Lu-type logstc estmato, the pncpal component Lu-type logstc estmato s ntoduced as an altenatve to the PCLR, ML and Lu-type logstc estmatos to deal wth the multcollneaty. The est of the pape s oganzed as follows. In Secton 2, the new estmato and some popetes of the new estmato ae pesented n Secton 3. A Monte Calo smulaton s gven n Secton 4 and some concludng emaks ae gven n Secton 5. 2 The new estmato The logstc egesson model s expessed by Agulea et al. (2006) n matx fom n tems of the logt tansfomaton as L X XTT Z whee T t,..., t p shows an othogonal matx wth ZVZ the odeed egenvalues of X VX TX VXT and dag,..., p,... p s. Then T and may be wtten as T T Tp and O O p and Zp VZ p T p X VXTp p. The Z whee ZVZ T X VXT matx and the vecto can be pattoned as Z Z Z p and p. The handlng of multcollneaty by means of PCR coesponds to the tanston fom the to the educed model L Z. model L X XTT XTpTp Z Z p p The by equaton () and PCR method we get the PCR estmato. Inan and Edoğan (203) poposed Lu-type logstc estmato (LTL) 4

5 whee ˆ k, d ( X VX ki ) ( X Vz d ˆ ) (2.) d and k 0 ae basng paametes. The pncpal component logstc egesson estmato (Agulea et al., 2006) s defned as we get We can wte (2.2) as follows: T T X VXT TX Vz (2.2) ˆ ˆ T TX VXT TX Vz T T ˆ (2.3) ML Then we can ntoduce a new estmato by eplacng ˆ * kd, ML wth ˆML n (2.3), and, ˆ, ˆ k d T T k d T T X VXT ki T X VXT di T X VXT T X Vz whee d and k 0 ae basng paametes. We call ths estmato as pncpal component Lu-type logstc egesson (PCLTL) estmato. (2.4) Remak: () It s obvous that ˆ, k d T TX VXT ki TX VXT di T ˆ, thus we can see the PCLTL estmato s a lnea combnaton of the PCLR estmato. (2) It s easy to obtan ˆ ˆ TX Vz, PCLR estmato (a) 0,0 T TX VXT ˆ ˆ p ML X VX X Vz, ML estmato (b) 0,0 ˆ k, d ˆ k, d ( X VX ki ) ( X Vz d ˆ ), LTL estmato. (c) p Thus, the new estmato n (2.4) ncludes the PCLR, ML and LTL estmatos as ts specal cases. 5 ML

6 The next secton we wll study the popetes of the new estmato. 3. The popetes of the new estmato Fo the sake of convenence, we show some lemmas whch ae needed n the followng dscussons. Lemma 3.. (Faebothe, 976, Rao and Tountenbug, 995) Suppose that M be a postve defnte matx, namely M 0, be some vecto, then M 0f and only fm. Lemma 3.2. (Baksalay and Tenkle,99) Let Cn p be the set of complex matces and H be the Hemtan matces. Futhe, gven L Cn p, n n * L, R L and L denote the conjugate tanspose, the ange and the set of all genealzed nveses, espectvely of L. Let A Hn n, a Cn and a 2 C n be lnealy ndependent, f aa a,, j,2 and j j A L, a R A. Let s a I AA I AA a2 a I AA I AA a Then A aa a2a 2 0 f and only f one of the followng sets of condtons holds: (a) 0,,,2, A a R A f f f 22 2 A 0, a R A, a R A: a, a sa A a sa s (b) A UU vv, a R A,, 2, va 0, f 0, f 0, (c) 22 f f f,

7 whee U: v shows a sub-untay matx, shows a postve scala. shows a postve defnte dagonal matx. Futhe, the condton (a), (b) and (c) denote all ndependent of the choce of A, A stands fo the genealzed nvese of A. To compae the estmatos, we use the mean squaed eo matx (MSEM) cteon whch s defned fo an estmato β as follows: MSEM(β ) = Cov(β ) + Bas(β )Bas(β ) whee Cov(β ) s the covaance matx of β, and Bas(β ) s the bas vecto of β. Moeove, scala mean squaed eo (SMSEM) of an estmato β s also gven as SMSE(β ) = t{msem(β )}. 3. Compason of the new estmato (PCLTL) to the ML estmato Fo (2.4), we can compute the asymptotc vaance of the new estmato as follows: whee S ( k) ki, S ( d) di. ˆ, ( ) ( ) ( ) ( ) Cov k d T S k S d S d S k T (3.) Usng (2.4), we get: ˆ E k, d T S ( k) S ( d) T (3.2) By T S ( k) T I T T kt S ( k) T (3.3) p p p Then we get the asymptotc bas of the new estmato as follows: ˆ, p p ( ) ( ) Bas k d T T d k T S k T We can get the asymptotc mean squaed eo matx of the new estmato as follows 7

8 ˆ, ( ) ( ) ( ) ( ) MSEM k d T S k S d S d S k T T T ( d k) T S ( k) T p p T T ( d k) T S ( k) T (3.4) p p Theoem 3.. Assume that d kand dk 0, then the new estmato s supeo to the ML estmato unde the asymptotc mean squaed eo matx cteon f and only f ( ) 2( ) ( ) p p p T k d k d I k d T T T. Poof: The asymptotc mean squaed eo matx of ML estmato MSEM ˆ X VX. (3.5) By O O p and T T, Tp, we may obtan X VX T T T TT T. (3.6) p p p Let us consde the followng dffeence ˆ ˆ, MSEM MSEM k d 2 2 ( ) 2( ) ( ) ( ) T S k k d I k d S k T T T T T ( k d) T S ( k) 2 p p p p p T S ( k) T ( k d) T S ( k) T T T p p ( k d) T T T S ( k) T. (3.7) p p Let S ( k) * S k d 0 0 p (3.8) 8

9 2 2 2( k d) I ( k d ) * 0 2 ( k d) 0 p (3.9) Now we can wte (3.7) as ˆ * * * ˆ, MSEM MSEM k d T S T T S T Thus MSEM ˆ MSEM ˆ k, d (3.0) s a nonnegatve defnte matx f and only f * * TT s a nonnegatve defnte matx. Usng Lemma 3., TT s a nonnegatve defnte matx f and only f * TT. Invokng the notaton of * n (3.9), we can pove Theoem Compason of the new estmato (PCLTL) to the PCLR estmato Theoem 3.2. Suppose that d k and dk 0, then the new estmato s bette than the PCLR estmato unde the asymptotc mean squaed eo matx cteon f and only f T 0. Poof: Suppose that k d n Equaton (3.4), then we get ˆ p p MSEM T T T T I T T I (3.) Now let us consde ˆ ˆ, MSEM MSEM k d 2 2 ( ) 2( ) ( ) ( ) T S k k d I k d S k T TT I p TT I p T T ( d k) T S ( k) T p p p p T T ( d k) T S ( k) T (3.2) 9

10 To apply Lemma 3.2, let A TBT, whee And p 2 2 B S ( k) 2( k d) I ( k d ) S ( k) (3.3) a T T I a2 Tp T p ( d k) T S ( k) T., When d k and dk 0, B s a postve defnte matx. Then we get the Mooe- Penose nveses of A s A, and AA TT TB T. Thus a R A f and only f a. Snce a 0, we cannot use pat (a) and (c) of Lemma 3.2, we can only apply pat 0 (b) of Lemma 3.2. Usng the defnton of s, we may obtan that s. On the othe hand, a2a A, whee Thus, we can easly obtan a R A: a 2 2 ( d k) T S ( k) 2( k d) I ( k d ) T (3.4) 2. Then Usng Lemma 3.2, we can get that the new estmato s supeo to the PCLR estmato unde the asymptotc mean squaed eo matx cteon f and only f a a A a a a a A a a o A 0. In fact, 2 2 0, so the new estmato s bette than the PCLR estmato unde the asymptotc mean squaed eo matx cteon f and only f A 0, that s 2 2 T 2( k d) I ( k d ) T 0 (3.5) 2 2 And T 2( k d) I ( k d ) T 0 f and only f T Compason of the new estmato (PCLTL) to the Lu-type logstc estmato Theoem 3.3. The new estmato s supeo to the Lu-type logstc estmato unde the asymptotc mean squaed eo matx cteon f and only f T 0. Poof: Put p nto (3.4), we get ˆ, ( ) ( ) ( ) ( ) MSEM k d TS k S d S d S k T p 0

11 2 ( k d) TS( k) T TS( k) T (3.6) Whee S( k) ki p and S(d) = Λ di p. Now we study the followng dffeence ˆ, ˆ, MSEM k d MSEM k d TS( k) S( d) S( d) S( k) T T S k S d S d S k T ( ) ( ) ( ) ( ) 2 ( k d) TS( k) T TS( k) T T ( d k) T S ( k) T p p T T ( d k) T S ( k) T (3.7) p p Suppose that C Tp DT, whee p and a d k TS k T D S ( k) S ( d) S ( d) S ( k) p p p p p 3 ( ) ( ), a2 Tp T p ( d k) T S ( k) T. We can apply pat (b) of Lemma 3.2. The Mooe-Penose nvese of C s C TpD Tp, and CC T T. So a R C, a RC : a p p 3, s and a 2 a 3 C, whee 2 3 T S ( k) S ( d) T p p p p p Then by Lemma 3.2, we obtan the new estmato s supeo to the Lu-type logstc estmato unde the asymptotc mean squaed eo matx cteon f and only f a a C a a o C In fact, a a C a a, so the new estmato s bette than the Lu-type logstc estmato unde the asymptotc mean squaed eo matx cteon f and only f C 0, that s T T 0. p p p

12 4. Monte Calo Smulaton Study In ths smulaton study, we study the logstc egesson model. In ths secton, we pesent the detals and the esults of the Monte Calo smulaton whch s conducted to evaluate the pefomances of the estmatos MLE, PCLR, and LTL estmatos and PCLTL. Thee ae seveal papes studyng the pefomance of dffeent estmatos n the bnay logstc egesson. Theefoe, we follow the dea of Lee and Slvapulle (988), Månsson, Kba and Shuku (202), Asa (207) and Asa and Genç (206) geneatng explanatoy vaables as follows 2 /2 x j z j zq (4.) whee,2,..., n, j,2,...,q and z j s ae andom numbes geneated fom standad nomal dstbuton. Effectve factos n desgnng the expement ae the numbe of explanatoy vaables q, the degee of the coelaton among the ndependent vaables 2 and the sample sze n. Fou dffeent values of the coelaton ρ coespondng to 0.8, 0.9, 0.99 and ae consdeed. Moeove, fou dffeent values of the numbe of explanatoy vaables consstng of p 4, 6, 8 and 2 ae consdeed n the desgn of the expement. The sample sze vaes as 200, 500 and 000. Moeove, we choose the numbe of pncpal components usng the method of pecentage of the total vaablty whch s defned as PTV = j= λ j p j= λ j 00. In the smulaton, PTV s chosen as 0.75 fo p = 4, 8 and 2 and 0.83 fo p = 6 (see Agulea et al. (2006)). The coeffcent vecto s chosen due to Newhouse and Oman (97) such that whch s a commonly used estcton, fo example see Kba (2003). We geneate the n 2

13 obsevatons of the dependent vaable usng the Benoull dstbuton Be whee x e e x such that x s the th ow of the data matx X. The smulaton s epeated fo 2000 tmes. To compute the smulated MSEs of the estmatos, the followng equaton s used espectvely: MSE 2000 c c (4.2) 2000 whee β c s MLE, PCLR, LTL, and PCLTL n the toleance s taken to be 6 0. th c eplcaton. The convegence We choose the basng paamete as follows: LTL: We efe to Asa (207) and choose d = mn { λ j } whee mn s the 2 (+λ j ) mnmum functon and k AM = p λ j d(+λ j α j2 ) p j=. λ j α j2 PCLTL: We use the same estmatos used n LTL. Table. Smulated MSE values of the estmatos when p = 4 n ρ MLE LTL PCLR PCLTL Table 2. Smulated MSE values of the estmatos when p = 6 n ρ MLE LTL PCLR PCLTL

14 Table 3. Smulated MSE values of the estmatos when p = 8 n ρ MLE LTL PCLR PCLTL Table 4. Smulated MSE values of the estmatos when p = 2 n ρ MLE LTL PCLR PCLTL Accodng to Tables -4, MSE of the MLE s nflated when the degee of coelaton s nceased. Smlaly, f we consde PCLR, ts MSE values ae also nflated fo nceasng values of the degee of coelaton. In geneal, nceasng the numbe of explanatoy vaables affects the estmatos negatvely, namely, ths stuaton makes MLE and PCLR less effcent such that MSE of MLE and PCLR ncease apdly. Howeve, LTL and PCLTL ae affected slghtly when the numbe of vaables changes. MLE and PCLR poduce hgh MSE values when the sample sze s low and the degee of coelaton s hgh. LTLT and PCLTLT ae obust to ths stuaton n almost all the cases. Inceasng the sample sze makes a postve effect on the estmatos n most of the stuatons. Howeve, thee s a degeneacy n ths popety especally when the degee of coelaton s hgh. LTL and PCLTL ae obust to the degee of coelaton.e. nceasng the degee of coelaton affects the pefomance of these estmatos postvely n most of the stuatons. 4

15 Oveall, LTL becomes the second-best estmato and the new estmato PCLTL has the lowest MSE value n all the stuatons consdeed n the smulaton. 5. Concluson In ths pape, we develop a new pncpal component Lu-type logstc estmato as a combnaton of the pncpal component logstc egesson estmato and Lu-type logstc estmato to ovecome the multcollneaty poblem. We have poved some theoems showng the supeoty of the new estmato ove the othe estmatos by studyng the asymptotc mean squaed eo matx cteon. Fnally, a Monte Calo smulaton study s pesented n ode to show the pefomance of the new estmato. Accodng to the esults, t seems that PCLTL s bette altenatve n multcollnea stuatons n the bnay logstc egesson model. Refeences Agulea, A. M., Escabas, M., and Valdeama, M. J. (2006). Usng pncpal components fo estmatng logstc egesson wth hgh-dmensonal multcollnea data. Computatonal Statstcs & Data Analyss, 50(8), Akdenz, F., and Eol, H. (200). Mean Squaed eo matx compasons of some based estmato n lnea egesson. Communcatons n Statstcs-Theoy and Methods. 32: Alheety. M.I., and Kba, B. M.G. (203). Modfed Lu-Type Estmato Based on ( - k) Class Estmato. Communcatons n Statstcs- Theoy and Methods. 2: Asa, Y. (207). Some new methods to solve multcollneaty n logstc egesson. Communcatons n Statstcs-Smulaton and Computaton, 46(4), Asa, Y., and Genç, A. (206). New Shnkage Paametes fo the Lu-Type Logstc Estmatos, Communcaton n Statstcs-Smulaton and Computaton, 45:3, Baye, M. R., and Paka, F. P. (984). Comnng dge and pcpal component egesson: a money demand llustaton. Communcatons n Statstcs-Theoy and Methods. 3: Baksalay, J. K., and Tenkle, G. (99). Nonnegatve and postve defnteness of matces modfed by two matces of ank one. Lnea Algeba and ts Applcaton. 5: Faebothe, R.W. (976). Futhe Results on the Mean Squae Eo of Rdge Regesson. Jounal of the Royal Statstcal Socety B, 38, Hoel, A.E., and Kennad, R. W. (970). Rdge egesson: based estmaton fo nonothogonal poblems. Technometcs. 2:

16 Huang, W.H., Q, J.J., Huang, N. T. (2009). Lu-type estmato fo lnea model wth lnea estctons. Jounal of System Scences and Mathematcal Scences. 29: Inan, D., and Edogan, B. E. (203). Lu-type logstc estmato. Communcatons n Statstcs-Smulaton and Computaton, 42(7), Kba, B. M. G. (2003). Pefomance of some new dge egesson estmatos. Communcatons n Statstcs-Smulaton and Computaton, 32(2), Lee, A. H., and Slvapulle, M. J. (988). Rdge estmaton n logstc egesson. Communcatons n Statstcs-Smulaton and Computaton, 7(4), Lesaffe, E., and Max, B.D. (993). Collneaty n Genealzed Lnea Regesson, Communcatons n Statstcs-Theoy and Methods, 22(7), Lu, K. (993). A new class of based estmate n lnea egesson. Communcatons n Statstcs- Theoy and Methods.22: L, Y.L., and Yang, H. (20). Two knds of estcted modfed estmatos n lnea egesson model. Jounal of Appled Statstcs. 38: Månsson, K., Kba, B. G., and Shuku, G. (202). On Lu estmatos fo the logt egesson model. Economc Modellng, 29(4), Newhouse, J. P. and Oman, S. D. (97). An evaluaton of dge estmatos. Rand Copoaton(P-76-PR), -6. Rao, C.R., and Toutenbug,H.(995). Lnea models: Least squaes and Altenatve. Spnge-Velag, New Yok. Saleh, A. M. E., and Kba, B. M. G. (203). Impoved dge egesson estmatos fo the logstc egesson model. Computatonal Statstcs, 28(6), Schaefe, R. L., Ro, L. D. and Wolfe, R. A. (984). A dge logstc estmato. Communcatons n Statstcs-Theoy and Methods, 3(), Xu, J. W., Yang, H. (20). On the estcted almost unbased estmatos n lnea egesson. Jounal of Appled Statstcs. 38: