AN OVERVIEW OF BIASED ESTIMATORS

Transcription

1 Jounal of Physical Science, Vol. 18(2), , AN OVERVIEW OF BIASED ESTIMATORS Ng Set Foong 1, Low Heng Chin 2 and Quah Soon Hoe 2 1 Depatment of Infomation Technology and Quantitative Sciences, Univesiti Teknologi MARA, Jalan Pematang Pauh, Pematang Pauh, Pulau Pinang, Malaysia 2 School of Mathematical Sciences, Univesiti Sains Malaysia, USM, Pulau Pinang, Malaysia *Coesponding autho: hclow@cs.usm.my/shquah@gmail.com Abstak: Pengangga pincang telah dicadangkan sebagai satu caa untuk meningkatkan kejituan anggaan paamete dalam model egesi apabila kekolineaan wujud dalam model tesebut. Sebab-sebab untuk menggunakan pengangga pincang telah dibincangkan dalam ketas keja ini. Satu senaai pengangga-pengangga pincang juga diumuskan dalam ketas keja ini. Abstact: Some biased estimatos have been suggested as a means of impoving the accuacy of paamete estimates in a egession model when multicollineaity exists. The ationale fo using biased estimatos instead of unbiased estimatos when multicollineaity exists is given in this pape. A summay fo a list of biased estimatos is also given in this pape. Keywods: multicollineaity, egession, unbiased estimo 1. INTRODUCTION When seious multicollineaity is detected in the data, some coective actions should be taken in ode to educe its impact. The emedies fo the poblem of multicollineaity depend on the objective of the egession analysis. Multicollineaity causes no seious poblem if the objective is to pedict. Howeve, multicollineaity is a poblem when ou pimay inteest is in the estimation of paametes. 1 The vaiances of paamete estimate, when multicollineaity exists, can become vey lage. Hence, the accuacy of the paamete estimate is educed. One obvious solution is to eliminate the egessos that ae causing the multicollineaity. Howeve, selecting egessos to delete fo the pupose of emoving o educing multicollineaity is not a safe stategy. Even with extensive examination of diffeent subsets of the available egessos, one might still select a subset of egessos that is fa fom optimal. This is because a small amount of

2 An Oveview of Biased s 90 sampling vaiability in the egessos o the dependent vaiable in a multicollinea data can esult in a diffeent subset being selected. 2 An altenative to egesso deletion is to etain all of the egessos, and to use a biased estimato instead of a least squaes estimato in the egession analysis. The least squaes estimato is an unbiased estimato that is fequently used in the egession analysis. When the pimay inteest of the egession analysis is in the paamete estimation, some biased estimatos have been suggested as a means to impove the accuacy of the paamete estimate in the model when multicollineaity exists. The ationale fo using biased estimatos instead of unbiased estimatos in a egession model when multicollineaity exists is pesented in Section 2 while an oveview of biased estimatos is pesented in Section 3. Some hybids of the biased estimatos ae pesented in Section 4. A compaison of the biased estimatos is pesented in Section THE RATIONALE FOR USING BIASED ESTIMATORS Suppose thee ae n obsevations. A linea egession model with p standadized independent vaiables, z1, z2,..., z p, and a standadized dependent vaiable, y, can be witten in the matix fom Y= Zγ+ ε (1) whee Y is an n 1 vecto of standadized dependent vaiables, Z is an n p matix of standadized independent vaiables, γ is a p 1 vecto of paametes, 2 ε is an n 1 vecto of eos such that ε ~N( 0, σ I n ) and In is an identity matix of dimension n n. 1 Let γˆ = ( ZZ ) ZY be the least squaes estimato of the paamete γ. The least squaes estimato, ˆγ, is an unbiased estimato of γ because the expected value of ˆγ is equal to γ. Futhemoe, it is the best linea unbiased estimato of the paamete, γ. Instead of using the least squaes estimato, biased estimatos ae consideed in the egession analysis in the pesence of multicollineaity. When the expected value of the estimato is equal to the paamete which is supposed to

3 Jounal of Physical Science, Vol. 18(2), , be estimated, then the estimato is said to be unbiased; othewise, it is said to be biased. The mean squaed eo of an estimato is a measue of the goodness of the estimato. The least squaes estimato (which is an unbiased estimato) has no bias. Thus, its mean squaed eo is equal to its vaiance. Howeve, the vaiance of the least squaes estimato may be vey lage in the pesence of multicollineaity. Thus, its mean squaed eo may be unacceptably lage, too. This would educe the accuacy of paamete estimate in the egession model. Although the biased estimatos have a cetain amount of bias, it is possible fo the vaiance of a biased estimato to be sufficiently smalle than the vaiance of the unbiased estimato to compensate fo the bias intoduced. Theefoe, it is possible to find a biased estimato whee its mean squaed eo is smalle than the mean squaed eo of the least squaes estimato. 1 Hence, by allowing fo some bias in the biased estimato, its smalle vaiance would lead to a smalle spead of the pobability distibution of the estimato. Thus, the biased estimato is close on aveage to the paamete being estimated THE BIASED ESTIMATORS Thee ae seveal biased estimatos that have been poposed as altenatives to the least squaes estimato in the pesence of multicollineaity. By combining these biased estimatos, some hybids of these biased estimatos ae fomed. Befoe pesenting the details of biased estimatos, a linea egession model which is in canonical fom is intoduced. Let λ be a p p diagonal matix whose diagonal elements ae eigenvalues of ZZ. The eigenvalues of ZZ ae denoted by λ1, λ2,..., λ p. Let the matix T = [ t1, t2,..., t p ] be a p p othonomal matix consisting of the p eigenvectos of ZZ, whee t j, j = 1, 2,..., p, is the j-th eigenvecto of ZZ. Note that matix T and matix λ satisfy TZZT = λ and TT = TT = I, whee I is a p p identity matix. By using matix λ and matix T, the linea egession model, Y = Zγ+ ε, as given by equation (1), can be tansfomed into a canonical fom Y= Xβ+ ε (2) whee X= ZT is an n p matix, β = T γ is a p 1 vecto of paametes and XX = λ.

4 An Oveview of Biased s 92 The least squaes estimato of the paamete β is given by βˆ = ( XX ) 1 XY (3) The least squaes estimato, ˆβ, is an unbiased estimato of β and is often called the Odinay Least Squaes (OLSE) of paamete β. In the pesence of multicollineaity, biased estimatos ae poposed as altenatives to the OLSE (which is an unbiased estimato) in ode to incease the accuacy of the paamete estimate. The details of these biased estimatos ae given below. The Pincipal Component Regession (PCRE) is one of the poposed biased estimatos. The PCRE is also known as the Genealized Invese. 3 6 Pincipal component egession appoaches the poblem of multicollineaity by dopping the dimension defined by a linea combination of the independent vaiables but not by a single independent vaiable. The idea behind pincipal component egession is to eliminate those dimensions that cause multicollineaity. These dimensions usually coespond to eigenvalues that ae vey small. The PCRE of paamete β is given by βˆ = T ˆ γ (4) 1 whee γˆ = T( TZZT ) TZY, is the PCRE of paamete γ, T = ( t1, t2,..., t ) is the matix of the emaining eigenvectos of ZZ afte we have deleted p of the columns of and it satisfies TZZT = λ = diag( λ, λ,..., λp ). T 1 2 The Shunken, o the Stein, is anothe biased estimato. It was poposed by Stein. 7,8 It is futhe discussed by Sclove (1968) 9 and Maye and Willke. 10 The Shunken is given by whee 0< s < 1. βˆ = sβ ˆ (5) s Tenkle poposed the Iteation. 11 The Iteation is given by βˆ = X Y (6) m, δ m, δ

5 Jounal of Physical Science, Vol. 18(2), , m i= whee the seies ( ) i X, δ = δ δ m I XX X, m = 0, 1, 2,..., 0 efes to the lagest eigenvalue. 1 0 < δ < and λ max λ max Tenkle stated that X m,δ conveges to the Mooe-Penose invese + 1 X = ( XX ) X of X. 11 Due to the fact that the least squaes estimato based on minimum esidual sum of squaes has a high pobability of being unsatisfactoy when multicollineaity exists in the data, Hoel and Kennad poposed the Odinay Ridge Regession (ORRE) and the Genealized Ridge Regession (GRRE). 12,13 The poposed estimation pocedue is based on adding small positive quantities to the diagonal of XX. The GRRE is given by βˆ = ( XX + K) -1 XY (7) K whee K = diag( ki ) is a diagonal matix of biasing factos ki > 0, i = 1, 2,..., p. When all diagonal elements of the matix, K, in the GRRE have values that ae equal to k, the GRRE can be witten as the ORRE. The ORRE is given by whee k > 0. βˆ = ( XX + ki) -1 XY (8) k Authos poved that the ORRE has a smalle mean squaed eo compaed to the OLSE. 12 The following existence theoem is stated in thei pape, Thee always exists a k > 0 such that the mean squaed eo of βˆ k is less than the mean squaed eo of ˆβ. Thee is also an equivalent existence theoem 12 fo the GRRE. The ORRE and the GRRE tun out to be popula biased estimatos. Many studies based on the ORRE and the GRRE have been done since the wok of Hoel and Kennad. 12,13 Some methods have been poposed fo choosing the value of k. 14,15 In 1986, Singh et al. 16 poposed the Almost Unbiased Genealized Ridge Regession (AUGRRE) by using the jack-knife pocedue. This estimato educes the bias unifomly fo all components of the paamete vecto. The AUGRRE is given as * K β = ( I ( XX + K) K ) β (9) -2 2 ˆ

6 An Oveview of Biased s 94 whee K diag( k ), k > 0, i = 1, 2,..., p. = i i In the case whee all diagonal elements of the matix, K, in the AUGRRE have values that ae equal to k, then we may wite the Almost 17 Unbiased Ridge Regession (AURRE) as whee k > 0. β k = ( I ( XX + ki) k ) β (10) -2 2 ˆ On the othe hand, Akdeniz et al. 18 (2004) deived geneal expessions fo the moments of the Lawless and Wang opeational AURRE fo individual egession coefficients. 18,19 Thee ae some othe biased estimatos developed based on the ORRE, such as the Modified Ridge Regession (MRRE) intoduced by Swindel. 20,21 and the Resticted Ridge Regession (RRRE) poposed by Saka 22,23 The MRRE and the RRRE ae given in equations (11) and (12), espectively. -1 b( k,b ) = ( X X + ki) ( X Y + k b ) (11) whee b is a pio mean and it is assumed that b β ˆ, k > 0. * -1-1 * β ( k) = [ I+ k( XX ) ] β (12) * whee k > 0, β = βˆ + ( XX ) R [ R( XX ) R ] ( Rβˆ) is the esticted least squaes estimato and the set of linea estictions on the paametes ae epesented by Rβ =. 4. HYBRIDS OF THE BIASED ESTIMATORS Biased estimatos have been poposed as altenatives to the OLSE when multicollineaity exists in the data. Majo types of the poposed biased estimatos ae the PCRE, the Shunken, the Iteation, the ORRE and the GRRE. Some studies have been done on combining the biased estimatos. Thus, some hybids of these biased estimatos have been poposed. Baye and Pake poposed the k Class which combined the techniques of the ORRE and the PCRE. 24 They poved that thee exists a k > 0

7 Jounal of Physical Science, Vol. 18(2), , whee the mean squaed eo of the k Class is smalle than the mean squaed eo of the PCRE. The k Class of paamete β is given by βˆ ( k) = T [ γˆ ( k)] (13) 1 whee p, k > 0, γˆ ( ) = ( + ) k T TZZT k I TZY is the k Class of paamete γ, T is the emaining eigenvectos of ZZ afte having deleted p of the columns of T and satisfying TZZT = λ = diag( λ1, λ2,..., λp). Liu intoduced a biased estimato by combining the advantages of the ORRE and the Shunken. 25 This new biased estimato is known as the Liu. The Liu can also be genealized to the Genealized Liu (GLE). The Liu and the GLE ae given in equations (14) and (15), espectively. whee 0< d < βˆ = ( XX + I) ( XY + dβˆ) d -1 βˆ = ( XX + I) ( XY + Dβˆ) D (14) (15) whee D = diag( di ) is a diagonal matix of the biasing factos, di, and 0< d i < 1, i = 1, 2,..., p. When all the diagonal elements of matix D in the GLE have values that ae equal to d, the GLE can be witten as the Liu. Liu showed that the Liu is pefeable to the OLSE in tems of the mean squaed eo citeion. 25 The advantage of the Liu ove the ORRE is that the Liu is a linea function of d. Hence, it is easy to choose d. Recently, Akdeniz and Oztuk deived the density function of the stochastic shinkage 26 paametes of the opeational Liu by assuming nomality. Some studies based on the Liu and the GLE have been done. Akdeniz and Kacianla intoduced the Almost Unbiased Genealized Liu (AUGLE). 21 This estimato is a bias coected GLE. When all the diagonal elements of the matix D in the AUGLE have values that ae equal to d, then the Almost Unbiased Genealized Liu can be witten as the Almost Unbiased Liu (AULE). 17 The AUGLE and the AULE ae given by equations (16) and (17), espectively. * D β = [ I ( XX + I) ( I D) ] β (16) -2 2 ˆ

8 An Oveview of Biased s 96 whee D diag( d ) and 0< d < 1, i = 1, 2,..., p. = i i whee 0< d < 1. d * β = [ I ( XX + I) (1 d) ] β -2 2 ˆ (17) Kacianla et al. intoduced a new estimato by eplacing the OLSE, ˆβ, * in the Liu, by the esticted least squaes estimato, β. 27 They called it the Resticted Liu (RLE) and it is given as ˆ -1 d ( ) ( β = XX + I XX + di) β * (18) whee * β = βˆ + ( XX ) R [ R( XX ) R ] ( Rβˆ) is the esticted least squaes estimato and the set of linea estictions on the paametes ae epesented by Rβ =. In 2001, Kacianla and Sakallioglu 28 poposed the d Class by combining the Liu and the PCRE. The d Class is a geneal estimato which includes the OLSE, the PCRE and the Liu as a special case. Kacianla and Sakallioglu have shown that the d Class is supeio to the PCRE in tems of mean squaed eo. 28 The d Class of paamete β is given by βˆ ( d) = T [ γˆ ( d)] (19) 1 whee p,0< d < 1, γˆ ( ) = ( + ) ( + ˆ d T TZZT I TZY d T γ ) is the d Class of paamete γ, γˆ = ( ) 1 T TZZT TZY is the PCRE of paamete γ, T is the emaining eigenvectos of ZZ afte having deleted p of the columns of and satisfying TZZT = λ = diag( λ, λ,..., λ ). T 1 2 p Table 1 displays a matix showing the biased estimatos and the hybids that have been poposed. The hybids that have been poposed ae the k Class, the Liu and the d Class. The Liu combines the advantages of the ORRE and the Shunken. The k Class combined the techniques of the ORRE and the PCRE while the d Class combined the techniques of the Liu and the PCRE. Thee ae some biased estimatos developed based on the ORRE, the GRRE, the Liu and the GLE. The MRRE, the RRE, the AUGRRE and the AURRE ae the biased estimatos developed based on the ORRE and the

9 Jounal of Physical Science, Vol. 18(2), , GRRE while the AUGLE, the AULE and the RLE wee developed based on the Liu and the GLE. The equations fo the biased estimatos pesented in Sections 3 and 4 ae summaized in Table 2. Table 1: Matix of the biased estimatos and the hybids. PCRE GRRE, ORRE Shunken Iteation GLE, Liu -k Class -d Class PCRE -k Class -d Class GRRE, ORRE Shunken Iteation MRRE, RRRE, AUGRRE, AURRE Liu GLE, Liu -k Class AUGLE, AULE, RLE -d Class 5. REVIEW ON THE COMPARISONS BETWEEN THE BIASED ESTIMATORS The compaisons among the biased estimatos as well as the OLSE ae found in seveal papes. Most of the compaisons wee done in tems of the mean squaed eo. An estimato is supeio to the anothe if its mean squaed eo is less than the othe.

10 Table 2: Summay of a list of estimatos. No. s* Equation Relevant Refeences 1 OLSE ˆ 1 β = ( XX ) XY Belsley PCRE βˆ = ˆ Tγ whee γˆ = ( ) 1 T TZZT TZY, is the PCRE of paamete γ, 3 Shunken T = [ t1, t2,..., t ] is the emaining eigenvectos of ZZ afte having deleted p of the columns of T and satisfying TZZT = λ = diag( λ, λ,..., λ ) 1 2 p βˆ = ˆ s sβ whee 0< s < 1 Massy 1965; Maquadt 1970; Hawkins 1973; Geenbeg 1975 Stein 1960; cited by Hocking et al. 1976; Sclove 1968; Maye & Willke Iteation βˆ m, δ = Xm, δy whee the seies ( ), δ = δ m i X δ m I XX X, i= 0 1 m = 0, 1, 2,..., 0 < δ < and λmax λ efes to the lagest eigenvalue max 5 GRRE ˆ -1 β = ( + ) K XX K XY whee K = diag( ki ) is a diagonal matix with biasing factos ki > 0, i = 1, 2,..., p 6 ORRE ˆ ( ) -1 β = + k XX ki XY whee k > 0 7 AUGRRE * βk -2 2 = ( I ( XX + K) K ) β ˆ whee K = diag( ki ), ki > 0, i = 1, 2,..., p 8 AURRE -2 2 β = ( ( + ) ) ˆ k I XX ki k β whee k > 0 Tenkle 1978 Hoel & Kennad 1970a,b Hoel & Kennad 1970a,b Singh et al Akdeniz & Eol 2003 (continue on next page)

11 Table 2: (continued) No. s* Equation Relevant Refeences 9 MRRE -1 b( k,b ) = ( X X + ki) ( X Y + k b ) Swindel 1976; cited by Akdeniz & whee b is a pio mean and it is Kacianla 1995 assumed that b β ˆ, k > 0 10 RRRE * -1-1 * β ( k) = [ I+ k( XX ) ] β Saka, 1992; cited whee k > 0, by Kacianla et al * β = βˆ+ ( XX ) R [ R( XX ) R ] ( Rβ) ˆ is the esticted least squaes estimato and the set of linea estictions on the paametes ae epesented by Rβ = 11 k Class βˆ ( ) [ ˆ k = T γ ( k)] Baye & Pake 1984 whee p, k > 0, γˆ ( ) = ( + ) 1 k T TZZT k I TZY is the k Class of paamete γ, T is the emaining eigenvectos of ZZ afte having deleted p of the columns of T and satisfying TZZT = λ = diag( λ, λ,..., λ ) 1 2 p 12 GLE -1 βˆ = ( + ) ( + ˆ D XX I XY Dβ ) whee D = diag( di ) is a diagonal matix of biasing factos d i and 0< d i < 1, i = 1, 2,..., p 13 Liu βˆ d -1 = ( XX + I) ( XY + dβˆ) whee 0< d < 1 14 AUGLE * -2 2 β = [ ( + ) ( ) ] ˆ D I XX I I D β, whee D = diag( di ) and 0< d i < 1, i = 1, 2,..., p 15 AULE * -2 2 β = [ ( + ) (1 ) ] ˆ d I XX I d β whee 0< d < 1 Liu 1993 Liu 1993 Akdeniz & Kacianla 1995 Akdeniz & Eol 2003 (continue on next page)

12 An Oveview of Biased s 100 Tabel 2: (continued) No. s* Equation Relevant Refeences 16 RLE ˆ -1 * β = ( + ) ( d XX I XX+ di) β Kacianla et al whee * β = βˆ + ( XX ) R [ R( XX ) R ] ( Rβˆ) is the esticted least squaes estimato and the set of linea estictions on the paametes ae epesented by Rβ = 17 d Class βˆ ( ) [ ˆ d = T γ ( d)] Kacianla & Sakallioglu whee p,0< d < 1, γˆ ( ) = ( + ) ( + ˆ d T TZZT I TZY d T γ ) is the d Class of paamete γ, γˆ = ( ) 1 T TZZT TZY is the PCRE of paamete γ, T is the emaining eigenvectos of ZZ afte having deleted p of the columns of T and satisfying TZZT = λ = diag( λ, λ,..., λ ) 1 2 p * No. 1 is an unbiased estimato while No.2 No. 17 ae biased estimatos Howeve, Singh et al. 16 compaed the GRRE and the AUGRRE in tems of bias. It is found that thee is a eduction in the bias of the AUGRRE when compaed with the bias of the GRRE in tems of absolute value. Table 3 gives a summay of the compaisons among the biased estimatos and the OLSE (which is an unbiased estimato) while Table 4 gives the elevant efeences of the compaisons. Hoel and Kennad compaed the OLSE, ˆβ, with the ORRE, βˆ k, and the GRRE, βˆ K. 12 It is found that thee exists a k > 0 such that the mean squaed eo of βˆ k is less than the mean squaed eo of ˆβ. Thee is also an equivalent 12 existence theoem fo the GRRE. Tenkle compaed the Iteation β ˆ m, δ and the OLSE, ˆβ. 11,29 It was found that the mean squaed eo of β ˆ m, δ is less than the mean squaed eo of ˆβ.

13 Jounal of Physical Science, Vol. 18(2), , Table 3: Summay of the compaisons among the estimatos. OLSE PCRE Shunken Iteation GRRE, ORRE UGRRE, AUREE MRRE RRRE -k Class Etimato GLE, Liu AUGLE, AULE RLE -d Class Etimato OLSE ii, iii i vii x viii ix xiv PCRE iii iv, x xiv Shunken iii Iteation iii xiii GRRE, ORRE v, vii vi x xiii, xv AUGRRE, AURRE xv MRRE xi RRRE -k Class GLE, Liu ix xii xiv AUGLE, AULE RLE -d Class In 1984, Baye and Pake 24 compaed the k Class, γˆ ( k), with the PCRE, γˆ. They showed that thee exists a k > 0 such that the mean squaed eo of γˆ ( k) is less than the mean squaed eo of γ fo 0 < p. ˆ

14 An Oveview of Biased s 102 Table 4: Refeences fo the compaisons among the estimatos. Refeences (i) Hoel & Kennad 1970a (ii) Tenkle 1978 (iii) Tenkle 1980 Compaison between the s (a) OLSE and ORRE (b) OLSE and GRRE (a) Iteation and OLSE (a) Iteation and OLSE (b) Iteation and ORRE (c) Iteation and Shunken (d) Iteation and PCRE (iv) Baye & Pake 1984 (a) PCRE and k Class (v) Singh et al (vi) Pliskin 1987 (vii) Nomua 1988 (viii) Liu 1993 (ix) Akdeniz & Kacianla 1995 (a) GRRE and AUGRRE (a) MRRE and ORRE (a) GRRE and AUGRRE (b) OLSE and AUGRRE (a) OLSE and Liu (a) GLE and AUGLE (b) OLSE and AUGLE (x) Saka 1996 (a) k Class and PCRE (b) k Class and OLSE (c) k Class and ORRE (xi) Kacianla et al (xii) Kacianla et al (xiii) Sakallioglu et al (xiv) Kacianla & Sakallioglu 2001 (xv) Akdeniz & Eol 2003 (a) MRRE and RRRE (a) RLE and Liu (a) ORRE and Liu (b) Iteation and Liu (a) d Class and PCRE (b) d Class and Liu (c) d Class and OLSE (a) GRRE and GLE (b) AUGRRE and AUGLE A compaison between the MRRE, b( k,b ), and the ORRE, β, was done by Pliskin. 30 A necessay and sufficient condition fo the mean squaed eo matix of ˆ β k minus the mean squaed eo matix of b( k,b ) to be positive semidefinite when both estimatos ae computed using the same value of k was developed. The autho suggested that eseaches who ae inclined to use the conventional ORRE should conside the MRRE if pio infomation is available. ˆ k

15 Jounal of Physical Science, Vol. 18(2), , ˆ d Liu made a compaison between the OLSE, ˆβ, and the Liu, β. 25 He showed that thee exists 0< d < 1 such that the mean squaed eo of is less than mean squaed eo of ˆβ. A compaison between the k Class and the OLSE, the PCRE, the ORRE was done by Saka (1996). 31 Necessay and sufficient conditions fo the supeioity of the k Class ove each of the othe thee estimatos using the mean squaed eo matix citeion wee obtained. Kacianla et al. 23 compaed the RRRE and the MRRE. They poved that the RRRE is supeio to the MRRE using the mean squaed eo matix citeion whethe the linea estictions ae tue o not. Kacianla et al. 27 intoduced the RLE and showed that the RLE is supeio in the scala mean squaed eo sense, to both the esticted least squaes estimato and to the Liu when the estictions ae indeed coect. They also deived conditions fo the supeioity of the RLE ove both the esticted least squaes estimato and the Liu when the estictions ae not coect. Kacianla and Sakallioglu made a compaison between the d Class, γˆ ( d), with the PCRE, γˆ, the Liu and the OLSE espectively. 28 They showed that thee exist 0< d < 1 such that the mean squaed eo of γˆ ( d) is less than the mean squaed eo of γˆ. The compaisons between the d Class and the Liu as well as the d Class with the OLSE show that which estimato is bette depends on the unknown paametes, the vaiance of the eo tem in the linea egession model and the choice of biased facto, d, in the biased estimatos. In addition, thee ae also seveal compaisons in tems of mean squaed eo which poduced simila conclusions. Tenkle compaed the Iteation with the ORRE, the Shunken and the PCRE espectively. 29 Nomua compaed the AUGRRE with the GRRE and the OLSE espectively. 32 Akdeniz and Kacianla made a compaison between the GLE and the AUGLE. 21 They also compaed the OLSE and the AUGLE. Recently, Sakallioglu et al. 33 compaed the Liu with the ORRE and the Iteation espectively. Akdeniz and Eol made a compaison between the GRRE and the GLE. 17 They also compaed the AUGRRE and the AUGLE. These compaisons showed that the bette estimato depends on the unknown paametes, the vaiance of the eo tem in the linea egession model and the choice of the biasing factos in biased estimatos. βˆ d

16 An Oveview of Biased s CONCLUSION Multicollineaity is one of the poblems that aise in egession analysis. Thus, multicollineaity diagnostics should be caied out to detect the poblem of multicollineaity in the data. The emedies fo the poblem of multicollineaity depend on the objective of the egession analysis. Multicollineaity causes no seious poblems if the objective is pediction. Howeve, multicollineaity is a poblem when the pimay inteest is in the estimation of the paametes in a egession model. In the pesence of multicollineaity, the minimum vaiance of the least squaes estimato may be unacceptably lage and hence educes the accuacy of the paamete estimates. Some biased estimatos have been suggested as a means to impove the accuacy of the paamete estimate in the model when multicollineaity exists. Thee ae seveal biased estimatos that have been poposed, such as the PCRE, the Shunken, the Iteation, the ORRE and the GRRE. In addition, the MRRE, the RRRE, the AUGRRE and the AUGRRE ae biased estimatos developed based on the ORRE and the GRRE. By combining these biased estimatos, some hybids of these biased estimatos, such as the k Class, the Liu, the GLE and the d Class ae obtained. Futhemoe, the AUGLE, the AUGLE and the RLE wee developed based on the Liu and the GLE. Fom most of the compaisons between the biased estimatos, 17,21,29,32,33 we find that the bette estimato depends on the unknown paametes and the vaiance of eo tem in the linea egession model as well as the choice of the biased factos in biased estimatos. Theefoe, thee is still oom fo impovement whee new classes of biased estimatos could be developed in ode to povide a bette solution. 7. REFERENCES 1. Rawlings, J.O., Pantula, S.G. & Dickey, D.A. (1998). Applied egession analysis A eseach tool. New Yok: Spinge-Velag. 2. Ryan, T.P. (1997). Moden egession methods. New Yok: John Wiley & Sons, Inc. 3. Massy, W.F. (1965). Pincipal components egession in exploatoy statistical eseach. Jounal of the Ameican Statistical Association, 60, Maquadt, D.W. (1970). Genealized inveses, idge egession, biased linea estimation and nonlinea estimation. Technometics, 12,

17 Jounal of Physical Science, Vol. 18(2), , Hawkins, D.M. (1973). On the investigation of altenative egessions by pincipal component analysis. Applied Statistics, 22, Geenbeg, E. (1975). Minimum vaiance popeties of pincipal component egession. Jounal of the Ameican Statistical Association, 70, Stein, C.M. (1960). Multiple egession. In I. Ikin (Ed.). Contibutions to pobability and statistics: Essays in Hono of Haold Hotelling. CA: Stanfod Univesity Pess, Hocking, R.R., Speed, F.M. & Lynn, M.J. (1976). A class of biased estimatos in linea egession. Technometics, 18, Sclove, S.L. (1968). Impoved estimatos fo coefficients in linea egession. Jounal of the Ameican Statistical Association, 63, Maye, L.S. & Willke, T.A. (1973). On biased estimation in linea models. Technometics, 15, Tenkle, G. (1978). An iteation estimato fo the linea model. Compstat, Hoel, A.E. & Kennad, R.W. (1970a). Ridge egession: Biased estimation fo non-othogonal poblems. Technometics, 12, Hoel, A.E. & Kennad, R.W. (1970b). Ridge egession: Applications to non-othogonal poblems. Technometics, 12, McDonald, G.C. & Galaneau, D.I. (1975). A Monte Calo evaluation of some idge type estimatos. Jounal of the Ameican Statistical Association, 70, Hemmele, W.J. & Bantle, T.F. (1978). An explicit and constained genealized idge estimation. Technometics, 20, Singh, B., Chaubey, Y.P. & Dwivedi, T.D. (1986). An almost unbiased idge estimato. Sankhya: The Indian Jounal of Statistics, 48(B), Akdeniz, F. & Eol, H. (2003). Mean squaed eo matix compaisons of some biased estimatos in linea egession. Communications in Statistics-Theoy and Methods, 32(12), Akdeniz, F., Yüksel, G. & Wan, A.T.K. (2004). The moments of the opeational almost unbiased idge egession estimato. Applied Mathematics and Computation, 153(3), Lawless, J.F. & Wang, P. (1976). A simulation study of idge and othe egession estimatos. Communications in Statistics-Theoy and Methods, A5, Swindel, B.F. (1976). Good idge estimatos based on pio infomation. Communications in Statistics-Theoy and Methods, A5, Akdeniz, F. & Kacianla, S. (1995). On the almost unbiased genealized Liu estimato and unbiased estimation of the bias and MSE. Communications in Statistics-Theoy and Methods, 24(7),

18 An Oveview of Biased s Saka, N. (1992). A new estimato combining the idge egession and the esticted least squaes methods of estimation. Communications in Statistics-Theoy and Methods, 21(7), Kacianla, S., Sakallioglu, S. & Akdeniz, F. (1998). Mean squaed eo compaisons of the modified idge egession estimato and the esticted idge egession estimato. Communications in Statistics, 27(1), Baye, M.R. & Pake, D.F. (1984). Combining idge and pincipal component egession: A money demand illustation. Communications in Statistics-Theoy and Methods, 13(2), Liu, K. (1993). A new class of biased estimate in linea egession. Communications in Statistics-Theoy and Methods, 22(2), Akdeniz, F. & Oztuk, F. (2005). The distibution of stochastic shinkage biasing paametes of the Liu type estimato. Applied Mathematics and Computation, 163, Kacianla, S., Sakallioglu, S., Akdeniz, F., Styan, G.P.H. & Wene, H.J. (1999). A new biased estimato in linea egession and a detailed analysis of the widely-analysed dataset on Potland cement. Sankhya: The Indian Jounal of Statistics, 61(B3), Kacianla, S. & Sakallioglu, S. (2001). Combining the Liu estimato and the pincipal component egession estimato. Communications in Statistics-Theoy and Methods, 30(12), Tenkle, G. (1980). Genealized mean squaed eo compaisons of biased egession estimatos. Communications in Statistics-Theoy and Methods, A9(12), Pliskin, J.L. (1987). A idge-type estimato and good pio means. Communications in Statistics-Theoy and Methods, 16(12), Saka, N. (1996). Mean squae eo matix compaison of some estimatos in linea egessions with multicollineaity. Statistics & Pobability Lettes, 30(2), Nomua, M. (1988). On the almost unbiased idge egession estimato. Communications in Statistics-Simulation and Computation, 17, Sakallioglu, S., Kacianla, S. & Akdeniz, F. (2001). Mean squaed eo compaisons of some biased egession estimatos. Communications in Statistics-Theoy and Methods, 30(2), Belsley, D.A. (1991). Conditioning diagnostics: Collineaity and weak data in egession. New Yok: John Wiley & Sons.