MSE Performance and Minimax Regret Significance Points for a HPT Estimator when each Individual Regression Coefficient is Estimated

Transcription

1 This article was downloaded by: [Kobe University] On: 8 April 03, At: 8:50 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, 37-4 Mortimer Street, London WT 3JH, UK Communications in Statistics - Theory and Methods Publication details, including instructions for authors and subscription information: MSE Performance and Minimax Regret Significance Points for a HPT Estimator when each Individual Regression Coefficient is Estimated Haifeng Xu a a Department of Statistics, School of Economics, Xiamen University and Wang Yanan Institute for Studies in Economics (WISE), Xiamen University, Xiamen, China To cite this article: Haifeng Xu (03): MSE Performance and Minimax Regret Significance Points for a HPT Estimator when each Individual Regression Coefficient is Estimated, Communications in Statistics - Theory and Methods, 4:, 5-64 To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

2 Communications in Statistics Theory and Methods, 4: 5 64, 03 Copyright Taylor & Francis Group, LLC ISSN: print/53-45x online DOI: 0.080/ MSE Performance and Minimax Regret Significance Points for a HPT Estimator when each Individual Regression Coefficient is Estimated HAIFENG XU Department of Statistics, School of Economics, Xiamen University and Wang Yanan Institute for Studies in Economics (WISE), Xiamen University, Xiamen, China. Introduction In this article, we consider a heterogeneous preliminary test (HPT) estimator whose components are the OLS and feasible ridge regression (FRR) estimators, and derive the exact formulae for the moments of the HPT estimator using mathematical method. Since we cannot examine the MSE of the HPT estimator analytically, we execute the numerical evaluation to investigate the MSE performance of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Furthermore, using the minimax regret criterion proposed by Sawa and Hiromatsu (973), we derive the optimal critical points of the preliminary F test. Our results show that the optimal significance points are greater than 9% and the optimal signicance points decrease as the denominator degrees of freedom of the preliminary F test statistic increases. Keywords Feasible ridge regression estimator; Heterogeneous preliminary test estimator; Minimax regret criterion; Optimal critical points. Mathematics Subject Classification 6J05; 6J07. Ridge regression estimators were proposed by Hoerl and Kennard (970) to deal with the problem of multicollinearity. If we accept some bias in trade for a reduction in variance, the ridge regression estimator can have smaller mean squared error (MSE) than the ordinary least squares (OLS) estimator in a certain region of parameter space. For this reason, no matter whether there is a problem of multicollinearity or not, the ridge regression estimator may be preferred to the OLS estimator in some cases. In applied regression analysis, we may be interested in a specific regression coefficient rather than all of them. For instance, Ohtani (997) showed the following Received June 4, 0; Accepted July 6, 0 Address correspondence to Haifeng Xu, Department of Statistics, School of Economics, and Wang Yanan Institute for Studies in Economics (WISE), Xiamen University, Xiamen, China; xhf984@hotmail.co.jp 5

3 MSE Performance and Optimal Significance Points 53 example. The import demand equation can be express as y t = 0 + x t + x t + t, where y t indicates the imports of a country at time t, x t indicates the real income (real GNP), and x t indicates the ratio of the import price to the domestic price (relative price). If our concern is to predict the impact of change in the real income on trade account (ceteris paribus), then the estimated coefficient (income elasticity) is desired to be as accurate as possible. From this viewpoint, Ohtani and Kozumi (996) and Ohtani (997, 000) examined the MSE performance of the Stein-rule (SR), positive-part Stein-rule (PSR) and minimum mean squared error (MMSE) estimators for each individual regression coefficient. Also, Srivastava and Giles (984) investigated the finite sample properties of a pre-test generalized ridge regression (PTGRR) estimator for each individual regression coefficient in a canonical form. Huang (999) showed the other situations that the estimation of each individual regression coefficient is important, and proposed a feasible ridge regression (FRR) estimator to estimate a specific regression coefficient. Also, several recent studies have focused on the FRR estimator, such as Ohtani (007, 008). Namba and Ohtani (009) considered a linear regression model and incorporated a preliminary test for an ineqality constraint on a regression coefficient into the ridge regression estimator proposed by Huang (999). They also derived the exact formula for the risk of the estimator under the asymmetric LINEX loss function and they compared the risk performance of the proposed ridge regression estimator with that of the OLS estimator using numerical evaluations. In the context of a linear regression model, a preliminary test for linear restrictions on regression coefficients can be used to choose the restricted or unrestricted estimators. Traditionally, the % or 5% level has been used as a level of a test. However, Sawa and Hiromatsu (973) proposed a minimax regret criterion to choose the optimal critical point of the preliminary test. Their results showed that the optimal critical point of the preliminary t test decreases slightly as the degrees of freedom increase, but it is nearly constant. In the context of estimating regression error variance, Toyoda and Wallace (975) presented that optimal significance levels of a preliminary F test vary somewhat but are close to / for most degrees of freedom and equal to / when the numerator and denominator of the degrees of freedom are equal. Also, Toyoda and Wallace, (976) derived optimal critical points for a preliminary test in regression using a minimum average relative risk criterion. Further, Brook (976) examined optimal critical points of the preliminary F test and their corresponding significance levels using the quadratic risk function and the regret function proposed by Sawa and Hiromatsu (973) for orthogonal and non orthogonal cases, respectively. His results suggested that if the F-statistic is more than two, the unrestricted estimator should be used. As is well known, the ridge regression estimator has smaller MSE than the OLS estimator over a certain region of parameter space. In particular, the MSE dominance of the FRR estimator over the OLS estimator holds when the individual regression coefficient is close to zero (see, for example, Huang, 999; Ohtani, 008). From this viewpoint, in this article, we consider the heterogeneous preliminary test (HPT) estimator whose components are the OLS and FRR estimators, and investigate the risk performance of the HPT estimator. We derive the exact general formula for the risk of the HPT estimator under the quadratic loss function The SR, PSR, and MMSE estimator are proposed by Stein (956), Baranchik (970), and Farebrother (975), respectively.

4 54 Xu (i.e., MSE), and we compare the risk of the HPT estimator with those of the FRR estimator and the usual OLS estimator using numerical evaluations. Furthermore, we derive the optimal critical points using the minimax regret criterion. The remainder of this article is organized as follows. In Sec., the model and estimators are presented. In Sec. 3, we derive the exact formula for the moments of the HPT estimator, and using the exact formula, we show the exact formula for the risk of the estimator under the quadratic loss function. In Sec. 4, by numerical evaluations, we compare the risk of the HPT estimator with those of the FRR estimator and the OLS estimator. In Sec. 5, using the minimax regret criterion proposed by Sawa and Hiromatsu (973), optimal critical points of the preliminary F test will be presented. Finally, we offer some concluding remarks in Sec. 6.. Model and Estimators Consider a linear regression model Y = X + = x + X + N0 I n () where Y is an n vector of observations on the explained variable, X = X is an n k matrix, x is an n vector of observations on an explanatory variable, and X is an n k matrix of observations on other explanatory variables. = is a k vector of regression coefficients for X, is a scalar coefficient for x which we have a special interest, is a k vector of coefficients for X, and is an n vector of error terms. As to the error terms, we assume that follows a normal distribution with location parameter 0 and scale parameter. Since X = X and =, the OLS estimator of is b = X X X Y, and the OLS estimator of is b = M x x M Y, where M = I n X X X X. The distribution of b can be written as b N M x () The FRR estimator of proposed by Huang (999) is given by ˆ = (x M x + s b ) x M y ( ) = M x b M x b + b (3) s where s = Y Xb Y Xb/ and = n k. Since the MSE dominance of the FRR estimator over the OLS estimator holds when the individual regression coefficient is close to zero, we consider the following HPT estimator whose components are the FRR and OLS estimators: ˆ c = IF<cˆ + IF cb ( ) = IF<c M x b M x b + b + IF cb (4) s

5 MSE Performance and Optimal Significance Points 55 where IA is an indicator function, c is a critical value of the pre-test, and F is the pre-test statisitic for the null hypothesis H 0 : = 0 against the alternative H 0: F = M x b s (5) Thus, ˆ c = ˆ if H 0 is accepted and ˆ c = b otherwise. 3. Moments and Risk of the HPT Estimator In this section, we derive the exact formulae for the moments of the MSE of the HPT estimator. As shown in the Appendix, the exact formulae for even and odd moments of ˆ c are and Eˆ c r = EIF < cˆ r + IF cb r [ ( ) = E IF<c M x b r ] M x b + b s r + EIF cb r = H r r c + H 0rc (6) Eˆ c r+ = EIF < cˆ r+ + IF cb r+ [ ( ) = E IF<c M x b r+ ] M x b + b s r b + EIF cb r b = J r + rc+ J 0rc (7) where [ ( H p q c + H p q c = E IF<c M x b M x b + s J p q c + J p q c = E [ + E IF c ( = x M x [ IF<c [ + E IF c = ( x M x ( M x b ) p b q ] ) p ] M x b + b s q ) q [ P p q c + P p q c ] (8) ) p b q b ] ( M x b M x b + s ( ) M x b p ] M x b + b s q b ) q [ T p q c + T p q c ] (9)

6 56 Xu and P p q c = q p T p q c = q p P p q c = q T p q c = q c/+c 0 c/+c 0 + / + q + i w i / + i/ t /+p+q+i t / + t p dt (0) + 3/ + q + i w i 3/ + i/ t /+p+q+i t / + t p dt () + / + q + i w i t /+q+i t / dt / + i/ c/+c () + 3/ + q + i w i t /+q+i t / dt 3/ + i/ c/+c (3) where = n k, = x M x /, w i = e / / i /i!. Using Eqs. (0) (3), the MSE of the ˆ c can be expressed as follows: MSEˆ c = Eˆ c = Eˆ c Eˆ c + = P c+ P 0 c T 0c+ T 0 0c + (4) Substituting Eqs. (0) (3) into (4), and differentiating with respect to c, we obtain: MSEˆ c c [ = P c + P ] [ 0 c T 0c + T ] 0 0c c c c c = + / + i + w i / + i/ [ + c + c + c/ + i / c i+/ + c i++/+ ] (5) From Eq. (5), the MSE performance of ˆ c depends on the sign of the following expression: [ ] + c + c (6) + c/ + i We consider the cases of = 0 and > 0 separately. When = 0, Eq. (6) is positive for all c 0 and the MSE of ˆ c is monotonically decreasing function of c. This result indicates that the MSE of ˆ c is minimized when c is infinity. In this case, the desirable estimator is the FRR estimator. Unfortunately, when > 0, there is the runing parameter i from zero to in Eq. (6), we cannot

7 MSE Performance and Optimal Significance Points 57 determine whether Eq. (6) is positive or negative. Thus, we examine the MSE of HPT estimators corresponding to some significance levels with that of the FRR estimator using numerical evaluations in the next section, and examine optimal critical points of the preliminary F test using the minimax regret criterion proposed by Sawa and Hiromatsu (973) in Sec MSE Performance of the HPT Estimator In this section, we execute the numerical evaluations to investigate the risk performance of the HPT estimator ˆ c, and compare with those of the FRR estimator ˆ and the usual OLS estimator b. In numerical evaluations, since MSEb = /x M x, we use the relative MSE defined as MSEˆ c MSEb = P c+ P 0 c T 0c+ T 0 0c + (7) The parameter values used in numerical evaluations are as follows: = n k = 4, 8, 6, 4, 60, 0; = various values. Also, c is chosen so that the pretest is conducted at %, 5% 0% 0%, and 50% significance levels. The numerical evaluations were executed on a personal computer, using a FORTRAN code. Whenever the increment of infinite series in the formulae of P p q c, T p q c, P p q c, and T p q c given in Eqs. (0), (), () and (3) becomes smaller than 0, the infinite series are judged to converge. Further, the integral in Eqs. (0) (3) are calculated using the Simpson s 3/8 rule with,000 equal subdivisions. Since we calculate the relative MSE defined in (7), ˆ c has smaller MSE than b when the relative MSE is smaller than unity. Since the results for = 4, 4, 0 is typical, we show the results for these cases and discuss in detail. Since ˆ c = ˆ when = 000, the HPT estimator becomes the FRR estimator in this case. Tables 3 show the relative MSE of the HPT and FRR estimators for = 4, 4, 0. As shown in the tables, for any given and, as increases, the values of the relative MSE of both ˆ c and ˆ climb up, decline and finally converge to unity from above. Also, as is clear in the tables, both ˆ c and ˆ have smaller MSE than b when (or ) is close to zero. It means that the HPT and FRR estimators are superior to the usual OLS estimator when the value of is not large. We also find out that the range of dominance of the HPT and FRR estimators over the usual OLS estimator get narrow as increases. Our numerical results also show that the relative MSE of ˆ c is smaller than that of ˆ in some cases of. For example, when = 4 and 000, the HPT estimators for all kinds of are smaller than the FRR estimator. Comparing Tables and, we can find that the range of dominance of the HPT estimator over the FRR estimator gets wide as increases. Further, as gets large, the range where the relative risk of the HPT estimator is smaller than that of the FRR estimator gets wide. However, when is close to zero, the relative risk of the HPT estimator increases as increases. It is clear that there exsits a trade-off between them. Therefore, it seems that a suitable level should be chosen to obtain the desirable HPT estimator in some sense.

8 58 Xu Table Relative MSE of ˆ c for = n k = 4 Significance level of the pre-test () Note: Since ˆ c = ˆ, when = 000, the HPT estimator becomes the FRR estimator. 5. Optimal Significance Points Using the Minimax Regret Criterion The relative risk performances of the HPT and FRR estimators are presented in Sec. 4. However, we cannot decide the optimal critical point of the preliminary F Table Relative MSE of ˆ c for = n k = 4 Significance level of the pre-test () Note: Since ˆ c = ˆ, when = 000, the HPT estimator becomes the FRR estimator.

9 MSE Performance and Optimal Significance Points 59 Table 3 Relative MSE of ˆ c for = n k = 0 Significance level of the pre-test () Note: Since ˆ c = ˆ, when = 000, the HPT estimator becomes the FRR estimator. Table 4 Optimal critical values, their percentage levels of the pre-test, and least favorable values of for = = n k c L U L = U % % % % % % Note: n = number of points in the sample space; k = number of parameters; indicates that R b c= Rˆ c; c = critical value of the pre-test of estimation; = percentage alpha level for the central F distribution, and degrees of freedom is and n k; L, U = optimal lower and upper values; L U = the minimized regret function values of ˆ b. test by the results of the risk performance. In this section, using the minimax regret criterion proposed by Sawa and Hiromatsu (973), an optimal value of c will be found by minimizing the maximum regret over all possible values of. Although we cannot show theoretically, our numerical results show that the MSE of the HPT estimator has a minimum value at c =+ for < and at c = 0 for, where is the value of such that Rˆ = R b. The values Rˆ and R b are quadratic risk functions (MSE) for the FRR (ˆ ) and the OLS estimator (b ), respectively.

10 60 Xu Figure. Risk function for different critical values when = n k = 4. of are shown in Table 4. The distances L = Lc and U = U c indicate the maximum additional risk of ˆ c above those of ˆ and b. This means that L and U are the regret for using ˆ c instead of ˆ or b. Figure shows the typical quadratic risk functions for different critical values. As is shown in Brook (976) and illustrated by Fig. that as c increases, L decreases and U increases. Our purpose is to control both L and U, and try to find an optimum value c = c which minimizes the maximum regret over all values of. Optimal values of c (says, c ) and percentage levels of the preliminary test (says, ) are presented in Table 4. From Table 4, it is clear that the outstanding property of the optimal critical value, c, is that it does not vary much. c only increases by about 0. as the denominator degrees of freedom of the preliminary test F distribution ( = n k) increases from 4 to 0. According to the results, c increases as. However, decreases as increases. Traditionally, the %, 5%, and 0% significance levels have been used in a testing procedure. If we use the 5% singnificance level in a pre-test in our context, it may lead to the possible large regret. However, if we use the optimal critical value in the sense of minimax regret, we can avoid obtaining such a large regret. This indicates that when at least a test is conducted to choose an estimator as a pretest, the indiscriminate use of traditional significance levels should be avoided in the sense of minimax regret. Table 4 also gives least favorable values, L, U, of the non centrality parameter of a noncentral F distribution. As increases, L increases and U slightly decreases. As increases, is steady at.. Further, L = U increases as increases. It indicates that the minimum value of the maximum regret function increases as the denominator degrees of freedom of the preliminary test F distribution () increases.

11 MSE Performance and Optimal Significance Points 6 6. Conclusion In this article, we considered the HPT estimator whose components are the OLS and FRR estimators, and derived the exact formulae for the moments of the HPT estimator using mathematical method. Since we cannot exmine the MSE performances analytically, we executed the numerical evaluation to investigate the MSE performance of the HPT estiamtor, and compared the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Furthermore using the minimax regret criterion proposed by Sawa and Hiromatsu (973), we derived the optimal critical points of the preliminary F test. Our results show that the optimal significance points are greater than 9%. As suggested by a referee, future research can be done to examine the sampling properties of a similar weighted average least squares (WALS) estimator combining the OLS and FRR estimators, which would probably exhibit superior properties to the pre-test estimator considered in this article. Appendix In this appendix, we derive the exact formulae for the case of the even and odd moments of the HPT estimator. First, we derive the first part of the even moments of ˆ c in Eq. (6). Let u = x M x b / and u = e e/ = s /, where = n k. Then, u and u, where f is the non central chi-square distribution with f degrees of freedom and non centrality parameter, and = x M x /. Using u and u, H p q c in Eq. (6) can be written as [ ( )( u H p q c = E I u / <c u u + u / ( = a q u K i u /u <c/ u + u / where K i = ) p ( x M x ) p u /+q+i ) q ] u q u / e u +u / du du (8) w i (9) +/+i / + i/ a = / M x, and w i = e / / i /i!. Making use of the change of variables, r = u + u and r = u /u, Eq. (8) can be written as a q K i 0 c/ r +/+q+i e r/ r /+p+q+i dr 0 + r +/+q+i r + / dr (0) p Also, making use of the change of a variable, z = r /, the first integral in Eq. (0) reduces to +/+p+q+i + / + q + i ()

12 6 Xu Further, making use of the change of a variable, t = r / + r, the second integral in Eq. (0) reduces to c/+c p t /+p+q+i t / + t p dt () 0 Substituting Eqs. () and () into Eq. (0), and performing some manipulations, we obtain H p q c in Eq. (6). Then, we derive the first part of the odd moments of ˆ c, J p q c in Eq. (7). Differentiating H p q c given in Eq. (8) with respect to, we can write H p q c ) q p w i G i p q c ( ) q [ ( x = q x M p M x x ( ) ] x + M x w i G i p q c ( ) x = M x H p q c ( = q x M x + q ( x M x where we define w = 0, and G i p q c = + / + q + i / + i/ ) q ( x M x c/+c 0 ) p ) w i w i G i+ p q c (3) t /+p+q+i t / + t p dt (4) On the other hand, using b and s, H p q c can be expressed as [ ( H p q c = E IF<c M x b M x b + s = F<c ( M x b M x b + s ) p b q ] ) p b q f b b f s s db ds (5) where f b b = exp [ b ] / x M x / M (6) x and f s s is the probability density function of s. Then differentiating H p q c given in Eq. (5) with respect to, we have H p q c = F<c ( x M x ) b

13 MSE Performance and Optimal Significance Points 63 ( M x b M x b + s ( x = M x ) J p q c ) p b q f b b f s s db ds ( x M x ) H p q c (7) Equating Eqs. (3) and (7), we obtain J p q c in Eq. (7). Similarly, using the same way, we can obtain H p q c and J p q c in Eqs. (6) and (7), respectively. Further, using H p q c and H p q c, we have the even moments of ˆ c in Eq. (6). Similarly, using J p q c and J p q c, we obtain the odd moments of ˆ c in Eq. (7). Acknowledgments I would like to thank Kazuhiro Ohtani for his guidance and suggestions. I would also like to thank Hisashi Tanizaki, Akio Namba, and an anonymous referee for their useful comments. References Baranchik, A. J. (970). A family of minimax estimators of the mean of a multivariate normal distribution. Ann. Mathemat. Statist. 4: Brook, R. J. (976). On the use of a regret function to set significance points in prior tests of estimation. J. Amer. Statist. Assoc. 7:6 3. Farebrother, R. W. (975). The minimum mean square error linear estimator and ridge regression. Technometrics 7:7 8. Hoerl, A. E., Kennard, R. W. (970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics : Huang, J. C. (999). Improving the estimation precision for a selected parameter in multiple regression analysis: an algebraic approach. Econ. Lett. 6:6 64. Namba, A., Ohtani, K. (009). Risk performance of a pre-test ridge regression estimator under the LINEX loss function when each individual regression coeffcient is estimated. J. Statist. Computat. Simul. 80:55 6. Ohtani, K., Kozumi, H. (996). The exact general formulae for the moments and the MSE dominance of the Stein-rule and positive-part Stein-rule estimators. J. Econometrics 74: Ohtani, K. (997). Minimum mean squared error estimation of each individual coefficient in a linear regression model. J. Statist. Plann. Infer. 6: Ohtani, K. (000). Shrinkage Estimation of a Linear Regression Model in Econometrics. New York: Nova Science Publishers. Ohtani, K. (007). On the evaluation of precision of estimation of a pre-test ridge regression estimator by bootstrap methods. Kobe Univ. Econ. Rev. 53: 7. Ohtani, K. (008). Small sample properties of a ridge regression estimator with an inequality constraint. Kobe Univ. Econ. Rev. 54:5 3. Sawa, T., Hiromatsu, T. (973). Minimax regret significance points for a preliminary test in regression analysis. Econometrica 4: Srivastava, V. K., Giles, D. E. A. (984). Exact finite-sample properties of a pre-test estimator in ridge regression. Austral. NZ J. Statist. 6:

14 64 Xu Stein, C. (956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Mathemat. Statist. Probab. Berkeley: University of California Press, pp Toyoda, T., Wallace, T. D. (975). Estimation of variance after a preliminary test of homogeneity and optimal levels of significance for the pre-test. J. Econometrics 3: Toyoda, T., Wallace, T. D. (976). Optimal critical values for pre-testing in regression. Econometrica 44: