A Semiparametric Generalized Ridge Estimator and Link with Model Averaging

Transcription

1 A Semiparametric Generalized Ridge Estimator and Link with Model Averaging Aman Ullah, Alan T.K. Wan y, Huansha Wang z, Xinyu hang x, Guohua ou { February 4, 203 Abstract In recent years, the suggestion of combining models as an alternative to selecting a single model from a frequentist prospective has been advanced in a number of studies. In this paper, we propose a new semi-parametric estimator of regression coe cients which is in the form of a feasible generalized ridge estimator by Hoerl and Kennard (970b), but with di erent biasing factors. We prove that the generalized ridge estimator is algebraically identical to the model average estimator. Further, the biasing factors that determine the properties of both the generalized ridge estimator and semi-parametric estimators are directly linked to the weights used in model averaging. These are interesting results for the interpretations and applications of both semi-parametric and ridge estimators. Furthermore, we demonstrate that these estimators based on the model averaging weights can have properties superior to the well-known feasible generalized ridge estimator in a large region of the parameter space. Finally, two empirical examples are also presented. Introduction The ordinary least squares (OLS) is a widely used estimator for the coe cients of a linear regression model in econometrics and statistics (Schmidt (976), Greene (20)). It is shown here that this estimator can also be obtained by estimating population moments (variances and covariances) of the economic variables involved in the regression by using empirical densities of their data sets. Further we propose a new estimator of the regression coe cients based on estimating population moments using Department of Economics, University of California, Riverside, address: aman.ullah@ucr.edu. y Department of Management Sciences, City University of Hong Kong, address: msawan@cityu.edu.hk. z Department of Economics, University of California, Riverside, address: huansha.wang@ .ucr.edu. x Academy of Mathematics and Systems Science and Center of Forecasting Science, Chinese Academy of Sciences, address: xinyu@amss.ac.cn. { Academy of Mathematics and Systems Science, Chinese Academy of Sciences, address: ghzou@amss.ac.cn.

2 a smooth kernel nonparametric density estimation. This proposed estimator, in contrast to the OLS estimator, is robust to the multicollinearity problem, and we refer to this as the semi-parametric (SP) estimator of regression coe cients. Further, although they are di erent, this SP estimator turns out to be in the form of a generalized ridge regression (GRR) estimator developed by Hoerl and Kennard (970b). Ridge regression (RR) (Hoerl and Kennard (970a, b)) is a common shrinkage technique in linear regression when the covariates are highly collinear, and among the various ridge techniques, the GRR estimator is arguably the one that has attracted the most attention. The GRR estimator allows the biasing factor, which controls the amount of ridging, to be di erent for each coe cient; when the biasing factors are the same for all coe cients, the GRR estimator reduces to the ordinary RR estimator. However, since the biasing factors are unknown the GRR estimator is not feasible. This is not the case for the SP estimator which is based on the information contained in the kernel density estimation of regressors, and hence the biasing factors are calculated using the data-based window-widths of regressors. Thus the SP estimator, in contrast to the GRR, is a feasible estimator. This SP estimator is compared with Hoerl and Kennard s (970b) feasible GRR (FGRR) estimator based on the rst step of a data-based iterative procedure for estimating the biasing factors. We note that Hemmerle and Carey (983) showed that the FGRR estimator is more e cient than the estimator based on the closed form solution of Hoerl and Kennard s iterative method. For more detail on GRR estimators, see Vinod and Ullah (98) and Vinod, Ullah and Kadiyala (98). Yet another technique independently developed, but closely related, to shrinkage estimation is model averaging, which is an alternative to model selection. While the process of model selection is an attempt to nd a single best model for a given purpose, model averaging compromises across the competing models, and by so doing includes the uncertainty associated with the individual models in the estimation of parameter precision. Model averaging also generates model against the selection of very poor models, and then holds the promise of reducing estimation risks. Bayesian model averaging (BMA) has long been a popular statistical technique. In recent years, frequentist model averaging (FMA) has also been garnering interest. A major part of this literature is concerned with ways of weighting models. For BMA, models are usually weighted by their posterior model probabilities, whereas FMA weighting models can be based on scores of information criteria (e.g. Buckland, Burnham and Augustin, (997); Claeskens, Croux and van Kerckhoven (2006); hang and Liang (20); and hang, Wan and hou (202). Other FMA strategies that have been developed include adaptive regression by mixing by Yang (200), Mallows model averaging (MMA) by Hansen (2007, 2008) (see also Wan, hang and ou (200)), optimal mean square error averaging by Liang, ou, Wan and hang (20) and Jackknife model averaging (JMA) by Hansen and Racine (202). As well, Hjort and Claeskens (2003) introduced a local misspeci cation framework for studying the asymptotic properties of FMA estimators. 2

3 Given these two independent, but parallel, developments of research in the ridge type shrinkage estimators and FMA estimators the objective of this paper is to explore a link between them. An initial attempt in this connection was in Leamer and Chamberlain (976) where they had noted a relationship between the ridge estimator and a model average estimator (which they called "search estimator"). However, they did not have the SP estimator introduced in our paper. Further, the ridge estimator considered in their paper is di erent from the one examined here unless the columns of the regressors matrix are orthogonal. More importantly, the Leamer and Chamberlain s (976) result does not always permit a connection between model averaging weights and ridge biasing factors. In contrast, in this paper, we establish an exact nite sample relationship between the biasing factors in the GRR estimator and the weights used in the FMA estimator. The biasing factors in the SP estimator is also indicated to be linked with the FMA weights. On the basis of these relationships, the selection of biasing factors in the GRR and SP estimators may be converted to the selection of weights in the FMA estimator. Our nding also implies that if the goal is to optimally mix the competing models based on a chosen criterion, e.g. Hansen s (2007) Mallows criterion, then there is always a GRR estimator that matches the performance of the resultant FMA estimator. We also demonstrate via a Monte Carlo study that the GRR estimators with biasing factors derived from the weights used for Hansen s (2007) MMA and Hansen and Racine s (202) JMA estimators perform well, in terms of risk, in a large region of parameter space. This paper is organized as follows. In Section 2, we present the SP and GRR estimators of the regression coe cients. In Section 3, we derive the exact algebraic relationship between the biasing factors of the SP and GRR estimators and the weights in the FMA estimator. Section 4 reports the results of a Monte Carlo study comparing the risks of SP, and FGRR estimators with biasing factors based on weights of the MMA and JMA estimators. Section 5 provides two empirical applications with the equity premium data in Campbell and Thompson (2008) and the wage data from Wooldridge (2003) using the SP and GRR estimators. Section 6 o ers some concluding remarks. 2 Semiparametric Estimator of Regression Coe cients Let us consider a population multiple regression model y = x + + x q q + u () = x 0 + u where y is a scalar dependent variable, x = [x ; :::; x q ] 0 is a vector of q regressors, is an unknown vector of regression coe cients, and u is a disturbance with Eu = 0 and V (u) = 2 : 3

4 If we minimize Eu 2 = E(y x 0 ) 2 with respect to we obtain = [Exx 0 ] Exy (2) where Exx 0 is a q q moment matrix of q variables with the j-th diagonal element and j; j 0 -th o diagonal elements, respectively, given by Ex 2 j = x 2 jf( )d ; j = ; :::; q; (3) x j E 0 = 0 f( ; 0)d d 0; j 6= j 0 = ; :::; q: 0 Suppose we have the sample observations fy i ; x i ; :::; x iq g; i = ; :::; n:then the population averages in (3) can be estimated by their sample averages as ^Ex 2 j = x 2 n ij; ^Exj 0 = n x ij x ij 0: (4) It is straightforward to note that these sample averages can be obtained as ^Ex 2 j = x 2 ^f( j )d = x 2 jd ^F ( ) (5) = x 2 ij n by using empirical distribution for ^F (). The results for ^E 0 similarly. Using (4) and (5) in (2), for all j and j 0,we get in (4) and ^E y = P n x ijy i =n follow ^ = ( ^Exx 0 ) ^Exy (6) = (X 0 X) X 0 Y where X is an n q matrix of observations on q variables, Y is an n vector of n observations and ^ is the well known ordinary least squares (OLS) estimator. Now we consider the estimation of Ex 2 j and E 0 by using a smooth nonparametric kernel density estimation instead of empirical distribution function. In this case, ~Ex 2 j = x 2 j f(x ~ j )d (7) = x 2 nh jk( x ij )d j h j = (x ij h j ij) 2 k( ij)d ij n = n = n ij ij (x 2 ij + h 2 j x 2 ij + h 2 j ij 2x ij h j ij)k( ij)d ij

5 where ~ f( ) = nh j P n k( xij xj h j ) is a kernel density estimator, ij = xij xj h j is a transformed variable, 2 = R v 2 k(v)dv > 0 is the second moment of kernel function, k( ij) is a symmetric second order kernel, and h j is window-width. For implementation, h j is selected by cross-validation, and kernel is chosen as normal or Epanechnikov quadratic function, see Pagan and Ullah (999). Similarly, it can easily be shown that ~E( 0) = 0 f(xj ~ ; 0)d d 0 (8) = = = n = n 0 nh j h j 0 nh j h j k( x ij ; x x ij0 j0 )d d 0 h j h j 0 0k( x ij )k( x x ij0 j0 )d d 0 h j h j 0 (x ij h j ij)(x ij 0 h j 0 ij 0)k( ij)k( ij 0)d ijd ij 0 ij ij 0 x ij x ij 0; and ~E( y) = x ij y i (9) n where we have used product kernels without any loss of generality and ij 0 = x ij 0 0 h j 0 :Also, ~ E(xj ) = n P n x ij = : Thus, using (7) to (9) in (2), a new estimator of can be introduced as ~ = ( ~ Exx 0 ) ~ Exy (0) = (X 0 X + D) X 0 Y where D = diag(d ; :::; d q ) is a diagonal matrix with the j-th (j = ; :::; q) element as d j = nh 2 j 2. We refer this estimator as the SP estimator: We note that both OLS and SP estimators are obtained by rst considering the population regression (), in which the regression coe cient vector depends on the population moments of vector x and scalar variable y, and then estimating these moments by two di erent methods with the help of sample data. These are the estimators of the regression coe cients in the sample linear regression model Y = X + U () where the sample is drawn from the population linear regression model (), and U is an n vector of random errors with EU = 0 and EUU 0 = 2 I n : By standard eigenvalue decomposition, we can write X 0 X = GG 0 ; where G is an orthogonal matrix and = diag( ; 2 ; :::; q ): 5

6 From Hoerl and Kennard (970a, b), the GRR estimator of is ^(K) = (X 0 X + GKG 0 ) X 0 Y (2) where K = diag(k ; k 2 ; :::; k q ) is a diagonal matrix with k j 0; j = ; :::; q: The kj 0 s are the biasing factors controlling the amount of ridging in ^(K): When k = k 2 = = k q = k; ^(K) is commonly called ordinary ridge regression estimator. We note that the SP estimator in (0) is in the form of GRR estimator but they are not the same. However, one may de ne an alternative SP-type estimator by taking the diagonal matrix D as the diagonal elements of GKG 0. Thus the elements of D can be determined from the biasing factors of GRR. Of course, if K = ki; then D = K and SP estimator is the same as the GRR estimator. De ne = XG and = G 0 : Then 0 = and model () may be reparameterized as Correspondingly, the GRR estimator of is Y = + U: (3) ^(K) = ( 0 + K) 0 Y = ( + K) 0 Y = B 0 Y (4) where B = ( + K) is a diagonal matrix. It is straightforward to show that ^(K) = G 0^(K): (5) Hence E(^(K) ) 0 (^(K) ) = E(^(K) ) 0 (^(K) ): (6) That is, the total MSE (Risk) of the GRR estimator of is the same as that of, and the matrix K that minimizes the risk of ^(K) also minimizes that of ^(K): 3 Link Between SP and Ridge Estimation with Model Averaging Let us consider the connection between the SP and GRR estimators and model averaging. For this, let us consider averaging across the sub-models Y = s s + U; s = ; 2; :::; S; (7) where s is a sub-matrix containing q s q columns of ;and s is the corresponding coe cient vector. From (7) we get that the OLS estimator ^ s = (s 0 s ) sy: 0 (8) 6

7 Let us write s = A s ; A s = (I qs : 0 qs(q q s)) or its column permutation is a q s q selection matrix: Similarly, we write s = A 0 s: The following model averaging (MA) estimator of is given by a weighted combination of coe cient estimators across the sub-models: ^(w) = SX w s A 0 s^ s (9) where w = (w ; w 2 ; :::; w S ) 0 is the weight vector with w s 0 and P S s= w s = : where We can equivalently write ^(w) in (9) as ^(w) = s= SX w s A 0 s(a s 0 A 0 s) A s 0 Y (20) s= = C 0 Y C = = SX [w s A 0 s(a s 0 A 0 s) A s ] (2) s= 0 w 0. C A 0 wq q and SX wj = w s I(j 2 s) (22) s= where I() is an indicator function which takes value if j 2 s and 0 otherwise, and s is a set comprising the column indices of included in the s-th sub-model. For example, if regressor matrix of the s-th sub-model comprises the rst, second and fourth columns of, then s = f; 2; 4g: In view of the relationship between wj and w s we can write (20) as ^(w ) = C 0 Y = ^(w) (23) where w = (w ; :::; w q) 0 : Comparing equations (4) and (20), we notice algebraic similarity between GRR estimator ^(K) = B 0 Y and MA estimator ^(w ) = C 0 Y: Clearly, ^(K) = ^(w ) if B = C; or more explicitly, w = ( + k ) (24). w q q = ( q + k q ) :. 7

8 This is the essence of the algebraic equivalence between the GRR and MA estimators. Note that 0 s depend on data, and w 0 s can be determined after MA weights w 0 s (Hansen (2007) and Hansen and Racine (202)) have been derived based on a given criterion. Then the biasing factors k 0 s of GRR estimator in (2) can be obtained from (24). As a simple illustration suppose that q = 2 in model () and the data observations are such that = and 2 = :5: Now the model averaging becomes a combination of S = 3 candidate sub-models. The rst two sub-models contain rst and second regressors respectively, and the full model contains both. Now, suppose the weights assigned are ^w = 0:5; ^w 2 = 0:2 and ^w 3 = 0:3 respectively. By (22): Then ^w = ^w 2 = 3X ^w s I( 2 s) = ^w + ^w 3 = 0:8 s= 3X ^w s I(2 2 s) = ^w 2 + ^w 3 = 0:5: s= ^k = ^w = 0:25 and ^k 2 = ^w = :5: Equation (24) also shows that when k = k 2 = = k q = 0 such that GRR estimator reduces to the OLS estimator, the MA estimator reduces to the OLS estimator in the full model. It should be mentioned that although (22) allows unique w j to be determined from the given values of w0 js, the converse need not to be true. Thus, while one can obtain unique GRR biasing parameters from the MA weights using (24), the reverse derivation of unique MA weights from the GRR biasing parameters is not always feasible. 4 A Monte Carlo Study The purpose of this section is to demonstrate via a Monte Carlo study the nite sample properties of the GRR estimator with biasing factors obtained on the basis of Mallows MA (MMA) and Jackknife MA (JMA) estimators by Hansen (2007) and Hansen and Racine (202), respectively. We denote these estimators as GRRM and GRRJ, respectively. From Hansen (2007), the weights of the MMA estimator are obtained by minimizing the quadratic form (Y ^(w)) 0 (Y ^(w)) + 2^ 2 tr(c 0 ) where ^ 2 = (Y ^ f ) 0 (Y ^ f )=(n q); and ^ f is the OLS estimator of in the full model. As to the JMA method in Hansen and Racine (202), the weights are determined by minimizing the leave-one-out least squares cross-validation function CV n (w) = (Y ^g(w)) 0 (Y ^g(w))=n, where ^g(w) = P S s= w s^g s and ^g s = (^g s ; :::; ^g ns ) 0 in which ^g is = x s0 i (Xs0 i Xs i ) X s0 i Y i where X s i and Y i denote matrices X s (X in the s-th submodel) and Y 8

9 with the i-th element deleted and x s i is the i-th observation in the s-th submodel. Following Hansen (2007) we assume that the candidate models in the model averaging are nested. We compare GRRM and GRRJ estimators of with those of FGRR estimator proposed by Hoerl and Kennard (970a, b), where ^ k j = ^ 2 =^ 2 j;f ; in which ^ j;f is the j-th element of ^ f : Also reported is the asymptotically optimal GRR estimator (AOGRR) where kj 0 s are obtained directly by minimizing Mallows criterion as a function of K; (Y ^(K)) 0 (Y ^(K)) + 2^ 2 tr(b 0 ), see Li (986, 987), Hansen (2007) and Wan, hang and ou (200). The GRR estimator corresponding to this K is asymptotically optimal because minimizing Mallows criterion as a function of K is equivalent to that as a function of w, due to the algebraic equivalence of GRR and MA. The last estimator is another SP estimator (SP), whose window-widths are obtained by minimizing Mallows criterion as a function of D; (Y ~(D)) 0 (Y ~(D))+2~ 2 tr(b 0 ) (see Li (986)), where the SP estimator ~(D) = (+G 0 DG) 0 Y = B 0 Y, in which B = ( + G 0 DG). Since no close form solution is available for the Mallows criterion, when implemented, we used the constraint optimization functions build in R.version For the AOGRR estimator, the initial values are set to be the k 0 s from the FGRR method. Regarding the SP, the initial values are set to be the "naive" window-widths, :06 x n =(4+q) : Our Monte Carlo experiments are based on two DGPs. DGP: y i = P q j= jx ij + e i ; x ij are iid N(0; ): The errors e i are uncorrelated with x 0 s so it is set to be iid N(0; ) and N(0; 25) respectively. This model is from Hansen (2007) with the values of j considered here as 0:707j 3=2 :Further, sample sizes taken are n = 50; q = and n = 50; q = 6. DGP2: In DGP 2, parameters and function are set as in DGP, but x 2i is set to be the sum of x 3i to x 50i plus an error which follows N(0; ) so that this DGP incorporates near-perfect collinearity. Under both DGP and DGP 2, 00 simulations are done. Further, normal kernel is selected, K() = (2) =2 exp[ 2 2 ]; and thus 2 = : The window-widths for the SP estimator in (0) are calculated under biased cross-validation as demonstrated in Scott and Terrell (987), and the windowwidths for the SP are as described above. Performance of the estimators is evaluated in terms of total MSE (risk), E(^ ) 0 (^ ); where ^ is an estimator. Simulation results are reported in Table. 9

10 Table : MSE for Each Estimator DGP Estimators = = 5 n = 50 n = 50 n = 50 n = 50 OLS FGRR GRRM GRRJ AOGRR SP SP OLS FGRR GRRM GRRJ AOGRR SP SP From the results we can observe that the SP estimator and the SP work better than all other estimators when the error variance is small. And when error variance is enlarged, the GRRM and GRRJ estimators would prevail. Although it is not reported here we nd that the GRRM and GRRM perform better when the error variance is moderately large. This is also consistent with our intuition model averaging gives more bene t when uncertainty is large. Also, we notice that AOGRR estimator is generating the similar results as the FGRR. This is perhaps because the choice of biasing factors in FGRR is known to be optimal for minimizing MSE, see Vinod, Ullah, and Kadiyala (98, p.363) and Hoerl and Kennard (970b, p.63). 5 Empirical Applications 5. Forecasting Excess Stock Returns The data is the same as in Campbell and Thompson (2008), Jin, Su and Ullah (202), and Lu and Su (202). In the monthly data from January 950 to December 2005 the total sample size is equal to 672. The dependent variable Y is the excess stock returns, which is de ned as the di erence between the monthly stock returns and the risk-free rate. We consider 2 regressors, default yield spread, treasury 0

11 bill rate, new equity expansion, term spread, dividend price ratio, earnings price ratio, long term yield, book-to-market ratio, in ation, return on equity, lagged dependent variable, smoothed earnings price ratio and the sub-models are nested. The independent variables are ordered in according to their correlation with the dependent variable. Thus, 3 candidate sub-models are generated with regressors fg; f; x g; :::; f; x ; x 2 ; :::; x 2 g; respectively. The in-sample estimation periods T are set to be 44, 80, 26, and 336 respectively. We de ne the out-of-sample R 2 as R 2 = P T t=t (Y t+ P T t=t (Y t+ ^Y t+ ) 2 Y t+ ) 2 where ^Y t+ ; Y t+ are the one-period-ahead prediction and historical average, respectively, using the sample of size T : 5.. Forecasting Results The out-of-sample R 2 are reported in Table 2. Table 2: Out-of-Sample R 2 Estimator T = 44 T = 80 T = 26 T = 336 OLS FGRR GRRM GRRJ AOGRR SP SP Several inferences could be drawn from the empirical results. First, OLS estimator could not beat the historical average, which is consistent with the nding in Welch and Goyal (2008) for this data set. Second, compared with the FGRR, AOGRR, and semiparametric estimators, the model averaging ridge estimators (GRRM and GRRJ) are doing much better since uncertainty is reduced when model averaging is incorporated. 5.2 Forecasting Wages The data is the same as in Wooldridge (2003), Hansen and Racine (202), and Lu and Su (202). The cross-sectional sample with size 526 is taken from the US Current Population Survey from year 976. The dependent variable is the logarithm of average hourly earnings. We consider 0 regressors, ordered as professional occupation, education, tenure, female, service occupation, married, trade,

12 SMSA, services, and clerk occupation based on their correlations with the dependent variable, and the sub-models are nested Forecasting Results Using the wage data, the out-of-sample R 2 ; as de ned in the earlier section with (T T, are reported in Table 3. ) replaced with Table 3: Out-of-Sample R 2 Estimator T = 00 T = 200 T = 300 T = 400 T = 500 OLS FGRR AOGRR GRRM GRRJ SP SP From the results above we observe that, in this cross-sectional dataset, all estimators are much better than the historical average. Yet when comparing the predictions, the SP as well as other estimators, are working better than the GRRM and GRRJ estimators. Though compared with MMA, JMA works better as seen in Hansen and Racine (202), our GRRM estimator using MMA works slightly better than the GRRJ using JMA in this cross-sectional dataset. The reason why here GRRM and GRRJ estimators are not working better, as what we had in the equity premium data application, is consistent with the simulation results. This is because R 2 = :509 for the wage data is much higher than that for the equity premium data, which is :097. Also, the coe cient of variation in the wage data (:23) is much smaller compared to that for the equity premium data (3:84). Thus, as our simulations suggest, for the small standard deviation case the GRRM and GRRJ estimators are outperformed by other estimators considered here. This is also the reason why in the equity premium data, the GRRM and GRRJ estimators prevail since the coe cient of variation for this data set is much higher. 6 Conclusion We have proposed a new SP estimator of regression coe cients which is in the form of the GRR estimator of Hoerl and Kennard (970b). However, in contrast to the GRR, the biasing factors in 2

13 our SP estimator are easily implemented by the window-width and the second moment of the kernel function used in the kernel density estimation. Another SP estimator is also proposed in which the selection of window-width is based on minimizing Mallows criterion. We also show that the GRR estimator is in fact a model average estimator, and there is an algebraic relationship between the biasing factors of the GRR estimator and the model average weights. Naturally, the GRR estimators that select the biasing factors based on this relationship have the similar properties as the corresponding model average estimator. This is an interesting nding for the future application and interpretations of the GRR and SP estimators. Our Monte Carlo and empirical results also demonstrate that some of the recently created weight choice schemes for model averaging can result in more accurate estimators (GRRM and GRRJ) than the SP, AOGRR, FGRR and OLS estimators for a moderate to large error variace. But for the case of a very small error variance SP estimators tend to perform better. Acknowledgements The authors are thankful to the participants of seminar in Concordia University, Montreal, especially John Galbraith. Aman Ullah s work was supported by the Academic senate, UCR. Alan Wan s work was supported by a General Research Fund from the Hong Kong Research Grants Council (Grant no. CityU-02709). Xinyu hang s work was supported by the National Natural Science Foundation of China (Grant nos. 704 and 27355). Guohua ou s work was supported by the National Natural Science Foundation of China (Grant nos and 026). 3

14 References Buckland, S.T., Burnham, K.P., Augustin, N.H. (997). Model selection: an integral part of inference, Biometrics 53(2): Claeskens, G., Croux, C., van Kerckhoven, J. (2006). Variable selection for logistic regression using a prediction-focused information criterion. Biometrics, 62(4): Campbell, J.Y., Thompson, S.B. (2008). Predicting excess stock returns out of sample: can anything beat the historical average? The Review of Financial Studies 2(4): Fan, J., Li, R., (200). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of American Statistical Association 96(456): Greene, W.H. (20). Econometric Analysis. New Jersey: Prentice Hall. Hansen, B.E. (2007). Notes and comments least squares model averaging. Econometrica 75(4): Hansen, B.E., Racine, J. (202). Jackknife model averaging. Journal of Econometrics 67(): Hemmerle, W.J., Carey M.B. (983). Some properties of generalized ridge estimators. Communications in Statistics: Computation and Simulation 2(3): Hoerl, A.E. (962). Application of ridge analysis to regression problems. Chemical Engineering Progress 58(3): Hoerl, A.E., Kennard. R.W., Baldwin, K.F. (975). Ridge regression: some simulations. Communications in Statistics 4(2): Hoerl, A.E., Kennard, R.W. (970a). Ridge regression: biased estimation for nonorthogonal problems. Technometrics 2(): Hoerl, A.E., Kennard, R.W. (970b). Ridge regression: application to nonorthogonal problems. Technometrics 2(): Hjort, N.L., Claeskens, G. (2003). Frequentist model average estimators. Journal of the American Statistical Association 98(464): Jin, S., Su, L., Ullah, A. (203). Robustify nancial time series forecasting. Econometric Reviews Forthcoming. Leamer, E.E., Chamberlain, G. (976). A Bayesian interpretation of pretesting. Journal of Royal Statistical Society 3(38): Li, K.C. (986). Asymptotic optimality of C L and generalized cross-validation in ridge regression with application to spline smoothing. The Annals of Statistics 4(3):0-2. Li, K.C. (987). Asymptotic optimality for C p,c L, cross-validation and generalized cross-validation: discrete index set. The Annals of Statistics 5(3):

15 Pagan, A., Ullah, A. (999). Nonparametric Econometrics. Cambridge: Cambridge University Press. Schmidt, P. (976). Econometrics. New York: CRC Press. Scott, D.W., Terrell C.R. (987). Biased and unbiased cross-validation in density estimation. Journal of American Statistical Association 82(400):3-46 Scott, D.W. (992). Multivariate Density Estimation: Theory, Practice and Visualization. New York: Wiley. Su, L., Lu, X. (202). Jackknife model averaging for quantile regressions. School of Economics, Singapore Management University. Working Paper. Vinod, H.D., Ullah, A. (98). Recent Advances in Regression Methods. New York: Marcel Dekker. Vinod, H.D., Ullah, A., Kadiyala, K. (98). Evaluation of the mean squared error of certain generalized ridge estimators using con uent hypergeometric functions. SANKHYA, Series B(43): Liang, H., ou, G., Wan, A.T.K., hang, X. (20). Optimal weight choice for frequentist model average estimators. Journal of the American Statistical Association 06(495): Wan, A.T.K., hang, X., ou, G. (200). Least squares model averaging by Mallows criterion. Journal of Econometrics 56(2): Yang, Y. (200). Adaptive regression by mixing, Journal of the American Statistical Association 96(454): Yuan,., Yang, Y. (2005). Combining linear regression models: when and how? Journal of Business and Economic Statistics 00(472): hang, X., Liang, H. (20). Focused information criterion and model averaging for generalized additive partial linear models. The Annals of Statistics 39(): hang, X., Wan, A.T.K., ou, G. (202). Model averaging by Jackknife criterion in models with dependent data. Journal of Econometrics Forthcoming. hang, X., Wan, A.T.K., hou, S.. (202). Focused information criteria, model selection, and model averaging in a Tobit model with a nonzero threshold. Journal of Business and Economic Statistics. 30():