Parametric Bootstrap Methods for Parameter Estimation in SLR Models

Transcription

1 International Journal of Econometrics and Financial Management, 2014, Vol. 2, No. 5, Available online at Science and Education Publishing DOI: /ijefm Parametric Bootstrap Methods for Parameter Estimation in SLR Models Chigozie Kelechi Acha * Department of Statistics, Michael Okpara University of Agriculture, Umudike, Abia State, Nigeria *Corresponding author: specialgozie@yahoo.com Received August 12, 2014; Revised August 25, 2014; Accepted September 01, 2014 Abstract The purpose of this study is to investigate the performance of the bootstrap method on external sector statistics (ESS) in the Nigerian economy. It was carried out using the parametric methods and comparing them with a parametric bootstrap method in regression analysis. To achieve this, three general methods of parameter estimation: least-squares estimation (LSE) maximum likelihood estimation (MLE) and method of moments (MOM) were used in terms of their betas and standard errors. Secondary quarterly data collected from Central Bank of Nigeria statistical bulletin 2012 from was analyzed using by S-PLUS softwares. Datasets on external sector statistics were used as the basis to define the population and the true standard errors. The sampling distribution of the ESS was found to be a Chi-square distribution and was confirmed using a bootstrap method. The stability of the test statistic θ was also ascertained. In addition, other parameter estimation methods like R 2, R 2 adj, Akaike Information criterion (AIC), Schwart Bayesian Information criterion (SBIC), Hannan-Quinn Information criterion (HQIC) were used and they confirmed that when the ESS was bootstrapped it turned out to be the best model with 98.9%, 99.9%, 84.9%, 85.4% and 86.7% respectively. Keywords: bootstrap, parametric models, parameter estimation, kernel density, quantile-quantile plots Cite This Article: Chigozie Kelechi Acha, Parametric Bootstrap Methods for Parameter Estimation in SLR Models. International Journal of Econometrics and Financial Management, vol. 2, no. 5 (2014): doi: /ijefm Introduction In econometrics, the goal of modeling is to deduce the form of the underlying process by testing the viability of such models. Once a model is specified with its parameters, and data have been collected, one is in a position to evaluate its goodness of fit, that is, how well it fits the observed data. Goodness of fit is assessed by finding parameter values of a model that best fits the data; a procedure called parameter estimation. There are basically three general methods of parameter estimation: least-squares estimation (LSE), maximum likelihood estimation (MLE), and method of moments (MOM).These parameter estimations have different methods under them and the methods that suit the study will be used, and they are least-squares estimation ordinary least square method (OLSM), maximum likelihood estimation secant method (SM), and method of moments cue method (CM), then for the kernel density and quantile-quantile plots - quadratic spectral method and also a parametric bootstrap method. There are also many sectors in the Nigerian economy namely; educational sector, agricultural sector, external sector statistics, financial sector and so on. External sector statistics is a vital aspect of the Nigerian economy. This is because whether seen from the point of view of export or import, it has fundamental implications for the economy. So it is therefore becomes expedient for any economy that wishes to grow to pay attention to changes in the external sector statistics. Therefore, the purpose of this study is to investigate and understand the parametric methods of the external sector statistics from the simple linear regression (SLR) under a variety of assessment conditions and to compare it with parametric bootstrap method in estimating its parameter. Datasets on export, import and gross domestic product (GDP) from Central Bank of Nigeria statistical bulletin 2012 from was used as the basis to define the population and the true standard errors. Up to twenty experiments were carried out by the author but the selection of the best model based on other evaluation criteria under several conditions of the original data set lead to four models. For the basis for comparison, the four models have thirty sample sizes and were bootstrapped fifty times with different parameter estimation methods. It is a well-known fact that import and export are strong determinants of economic growth and as the nation works hard towards growing its economy into one of the twenty leading economies through vision 2020 goals, there is need to establish a platform for the prediction and forecasting of the Nigeria economy. This necessitates the need for the study to establish the models, the approximate distribution, stability of the test statistics, kernel density and qq plot of the external sector statistics in Nigeria. 2. Literature Review

2 International Journal of Econometrics and Financial Management 176 The bootstrap is a statistical technique used more and more widely in econometrics. Generally, it falls in the broader class of resampling methods called the parametric and nonparametric bootstrap. Bootstrapping is the practice of estimating properties of an estimator by measuring those properties when sampling from an approximating distribution. One standard choice for an approximating distribution is the empirical distribution of the observed data. In the case where a set of observations can be assumed to be from an independent and identically distributed population, this can be implemented by constructing a number of resamples of the observed dataset and of equal size to the observed dataset, each of the bootstrap methods is obtained by random sampling with replacement for more details, see, Efron (2000); Efron and Tibshirani (1993). There are many resampling methods like simple, double, weighted, wild, recursive, segmented, residual, parametric, nonparametric and so on and their introductory aspects can be seen in Lahiri (2006), Xu (2008) and Quenouille (1956). In certain circumstances, such as regression models with independent and identically distributed error terms, appropriately chosen bootstrap methods generally work very well. Bootstrap methods are often used as an alternative to inference based on parametric assumptions, or to examine the stability of the test statistic θ or when those assumptions are in doubt, or where parametric inference is impossible or requires very complicated formulas for the calculation of standard errors, confidence interval, constructing hypothesis tests, etc. There are many bootstrap methods that can be used for econometric analysis especially in regression and the have been discussed extensively by Efron (2000), Efron and Tibshirani (1993), Gonzalez-Manteiga and Crujeiras (2008), Freedman (1981) Good (2004), Hall and Maiti (2006), Hall, Lee and Park (2009), Mahiane and Auvert (2010), Paparoditis, and Politis, (2005). Here the parametric bootstrap and the three basic estimation parameters will be used. The purpose of this study is to investigate the performance of the external sector statistics in the Nigerian economy. The external sector statistics is very crucial and strong determinants of economic growth. Therefore, it is very necessary to establish the models, the approximate distribution, stability of the test statistics, kernel density and qq plot of the external sector statistics in Nigeria. To the best of my knowledge they have not been established, and they are very important as the nation works hard toward attaining vision 2020 goals. This study is carried out by using a parametric bootstrap method and by comparing the parametric models and parametric bootstrap method in the regression analysis in terms of their betas and standard errors. Datasets on export, import and gross domestic product (GDP) was used as the basis to define the population and the true standard errors. To buttress the main purposes of bootstrap; suppose we have a set of observations {x 1, x 2, x n } and a test statistic θ. The resampling methods are often useful to examine the stability of θ and compute the estimations for the standard error of θ, where the distribution of θ is unknown, or that consistent estimations from the standard error of θ are not available, in this case the resampling (bootstrap) methods are especially useful. 3. Research Methodology The parametric methods and the parametric bootstrap method that will be used in this study are the three basic parameter estimations in simple linear regression and the parametric bootstrap method. Here, we shall discuss in details our approach: least-squares estimation ordinary least square method (OLS), maximum likelihood estimation (MLE) secant method, and method of moments (MOM) cue method, then for the kernel density and quantile-quantile plots - quadratic spectral method. In addition, bootstrap method, other information criteria methods; Akaike Information criterion (AIC), Schwart Bayesian Information criterion (SBIC), Hannan- Quinn Information criterion (HQIC) will be established since all the parameter estimation methods together define the performance of an estimator. REGRESSION AND PARAMETRIC BOOTSTRAP The data set x for a classic linear regression model consist of n points x 1, x 2,, x n, where each x i is itself a pair, say ( ) x c,y i = i i (3.1) where c i is a 1xp vector c i = (c i1, x i2,, c ip ) called the covariate vector or predictor, while y i is a real number called the response. Let µ i indicate the conditional expectation of ith response y i given the predictor c i. The probability structure of the linear model is usually expressed as y i=ciβ + i i = 1,2,.,n (3.2) the error terms ϵ i in (1.4) are assumed to be a random sample from an unknown error distribution F having expectation 0, ( β1 β2 βp) F( ) F,,, = E = 0 (3.3) We want to estimate the regression parameter vector β from the observed data (c 1, y 1 ), (c 2, y 2 ),, (c n, y n ). let C be the n x p matrix with ith row c i (the design matrix), and let y be the vector ( y 1, y 2,, y p ) T. then the least squares estimate is the solution to the normal equations T ( ) 1 ˆ T = C C C y (3.4) β where C is of full rank p. Furthermore, xi s from the original sample was used and the associated yi s were generated by a random draw y xi from F(xi, ˆ θ ). That is, N bootstrapped samples of x, call each of them xi* was drawn, where i = 1, 2, N, and generate the associated y s by random draws from F(xi*, ˆ θ ). Furthermore, bootstrap can be apply to more general regression to make a statistical inference. Efron and Tibshirani, (1993), suggested bootstrap algorithm for estimating standard errors and bias from regression models as shown below; 1. Select B independent bootstrap samples x* 1, x* 2,, x* B each consisting of n data values drawn with replacement from x, for estimating a standard error, the number B will ordinarily be in the range Evaluate the bootstrap replication corresponding to each bootstrap sample,

3 177 International Journal of Econometrics and Financial Management b ( ) ( ) ˆ* θ b = s x* b = 1,2,,B. (3.5) 3. The bootstrap estimate of standard error is the standard deviation of the bootstrap (B) replications: where B (3.6) b= 1 se * b = s ( X ) s ( ) ( B 1) boot B * b ( ) = ( ) s s X B. b= 1 4. Presentation, Results and Analysis The main aim of this section is to analyze the results in order to make necessary policy deductions from them. PARAMETRIC CASE A. Case 1: parametric model I applied the three methods of parameter estimation in regression namely (OLS), (MLE) and (MOM)] on the original data set. Tested Hypothesis Ho: Nigerian external sector does not contribute significantly to economic growth. SLR EQUATION ESTIMATED: GDPt = bo + b1im + b2ex + e From R-Statistical package the OLS model (A311) obtained is GDPt = 4.235e e + 00IM e + 00EX+e (4.1) Standard error (2.055e+05) (3.235e-01) (1.841e-01) From R-Statistical package the MLE model (A312) obtained from the hessian matrix is GDPt = b b 2+e (4.2) Standard error ( ) ( ) ( ) From R-Statistical package the GMM (A313) model obtained is GDPt = e e+00zm zm 2 +e (4.3) Std. Error (9.5280e+04) ( e-01) (3.7148e-01) B. Case 2: parametric bootstrap model Tested Hypothesis Ho: Nigerian external sector does not contribute significantly to economic growth. SLR EQUATION ESTIMATED: GDPt = bo + b1im + b2ex + e From R-Statistical package the OLS model (B311) obtained when n = 50 is GDPt=5.145e e + 00EX e + 00IM+e (4.4) Std. Error (1.585e+05) (1.235e-01) (2.129e-01) Where e=error term, t=1,2,,30, IM=b2=zm2=import and EX=b1=zm1=export in all the equations. All the models are from the program developed in S- plus (see Acha, C. K. (2014) unpublished dissertation for the complete program). The models in case 1&2, which are, equation 4.1, 4.2, 4.3, and 4.4 indicate positive relationship between GDP and Nigeria external sector. The positive sign of the external sector rate suggests that import and export correlates with the economy in Nigeria as suggested by Theory, even though, export rate is higher. The A311 model using OLSM and A312 model using CM are exactly the same value when approximated but there is a little difference in A313 model using SM. Moreover, A311 model was the best among the three models when compared in terms of their coefficients of the independent variables (β s) and standard errors. Since the A311 model was the best among the three models, it was further bootstrapped. In fact, comparing in terms of their coefficients of the independent variables of the Parametric models and Parametric bootstrapped models using Betas (β s) and standard errors to ascertain the best under several conditions The bootstrapped model (B311) turned out to be better than A311 model. C. CASE 3: Selection of the Best Model Based on other Evaluation Criteria under several conditions of the original data set gotten from the OLS, MLE, MOM and bootstrapped models of the SLR equation. Table 1. Selection of the Best Model Based on other Evaluation Criteria R 2 R 2 adj AIC HQIC SBIC Convergence Parametric OLS Parametric MLE Parametric MOM Parametric BOOT * * 84.9* * * 0 It is pertinent to note that when the coefficient of determination (adjusted R 2 ) is high, it shows that the model is a reasonable fit of the relationship among the variable. It also confirms its efficiency in prediction. The model hence explains 98% of the behavior of external sector in Nigeria. Since the probability (F-statistic) of 0.00 is less than the chosen 5% significance level we cannot accept the null hypotheses 1 and 2. The alternative hypotheses that the external sector in Nigerian economy is significant are accepted instead. The best set of parameters found in parametric bootstrapped information criteria and the function minimizes with first argument the vector of parameters over which minimization is to take place and the result must be a scalar. However, among all the Models based on other evaluation criteria, parametric bootstrapped information converges at zero, which indicates successful completion of the bootstrap analysis. D. kernel density and quantile-quantile plots of the original and bootstrapped data set. To identify the sampling distribution of a statistic θ (the mean): Here the shape, center and spread of the kernel density and quantile-quantile plots of the original data set and the original when bootstrapped are being considered. The kernel density and the qq-plots in Figure 1 & Figure 2 shows that there is variation among the means. It also

4 International Journal of Econometrics and Financial Management 178 shows that the distribution possess the following characteristics; 1. It is not symmetric around the mean. 2. It is skewed to the right in small samples. 3. The mean value and the variance are not equal to zero and one respectively. Hist of GDP Data Sample Qua 0. QQ-Plot for GDP Data 1e+07 2e+07 3e Hist of Export Data QQ-Plot for Export Data 5e+06 1e+07 Hist of Import Data QQ-Plot for Import Data -2e+06 2e+06 4e+06 6e+06 8e+06 1e+07 Figure 1. The kernel density and quantile-quantile plots of the original data set from the Nigerian external sector statistics Rnorm Hist of GDP Sample Qu -1e+07 Rnorm GDP QQ-pl -2e+07 2e Rnorm Hist of Exp Sample Quan Rnorm Export QQ Rnorm Hist of Imp -2e+06 Rnorm Import QQ- -4e+06 4e+06 Figure 2. The kernel density and quantile-quantile plots of the bootstrap data set from the Nigerian external sector statistics To identify the sampling distribution of a statistic θ (the mean): Here the shape, center and spread of the kernel density and quantile-quantile plots of the original data set and the original when bootstrapped are being considered. The kernel density and the qq-plots in figure 1 & 2 shows that there is variation among the means. It also shows that the distribution possess the following characteristics; 1. It is not symmetric around the mean. 2. It is skewed to the right in small samples. 3. The mean value and the variance are not equal to zero and one respectively. Statistical theory shows that any distribution with the following properties is a Chi-square distribution. Therefore, the approximate distribution of the external sector statistics in Nigeria is a Chi-square distribution and it is denoted by χ 2.

5 179 International Journal of Econometrics and Financial Management 5. Conclusion The key/main findings of this paper are summarized as follows; The parametric bootstrapped model (B311) turned out to be better than the parametric model (A311) under various conditions. The parametric estimation methods, parametric bootstrapped method and the information criteria confirmed that model A311 is the best. The approximate distribution of the external sector statistics in Nigeria has been found to be a Chisquare distribution. The stability of the test statistics was ascertained. The kernel density and qq plot also shows that when the ESS was bootstrapped, it gave a better plot and less outliers. Gross domestic product (GDP) and export seems to be highly positively correlated even though import is also positively correlated. The high correlation of export and GDP is very surprising since all the things we use in Nigeria are imported. This influence on export should be from the oil sector. In the real sector, the importance of export in development cannot be over emphasized. This is why the economy like Nigeria s in dire need of development must boost the non-oil export while not discouraging import. Apart from the result showing a good fit, the external sector statistics sampling distribution in Nigeria of the original dataset and also when bootstrapped was ascertained. One of the policy implications is that the established models, distributions of ESS form a good platform for the prediction and forecasting of the Nigeria economy. This is very necessary as the nation works toward attaining vision 2020 goals because import and export are strong determinants of economic growth. The limitations of resampling methods are that if the observations in the original sample are not exchangeable, the resampling sometimes does not provide the true distribution, but with the resample we can estimate the empirical distribution function (EDF), that will provide appropriate approximation of the true distribution. In spite of the limitations and the need for more research regarding the bootstrap methods for estimating the external sector statistics in Nigeria especially the non-oil sector. References [1] Efron, B. (2000). The bootstrap and modern statistics. J. Am. Statist. Assoc. 95, [2] Efron, B. and Tibshirani, R.J. (1993). An Introduction to the Bootstrap. Chapman and Hall, New York. [3] Freedman, D.A. (1981) Bootstrapping regression models, Ann. Statist. 6, [4] Gonzalez-Manteiga, W. and Crujeiras, R.M. (2008). A review on goodness-of-fit tests for regression models. Pyrenees International Workshop on Statistics, Probability and Operations Research: SPO 2007, [5] Good, P. (2004). Permutation, Parametric, and Bootstrap Tests of Hypotheses. 3rd Edition. Springer-Verlag, New York. [6] Hall, P. and Maiti, T. (2006). On parametric bootstrap methods for small area prediction. Journal of the Royal Statistical Society. Series B 68, [7] Hall, P. Lee, E.R. and Park, B.U. (2009) Bootstrap-based penalty choice for the Lasso achieving oracle performance. Statistica Sinica, 19, [8] Lahiri, S. N. (2006). Bootstrap Methods: A Review. In Frontiers in Statistics (J. Fan and H.L. Koul, editors) , Imperial College Press, London. [9] Mahiane, S. G., Nguema, E.-P. Ndong, Pretorius, C., and Auvert, B. (2010). Mathematical models for coinfection by two sexually transmitted agents: the human immunodeficiency virus and herpes simplex virus type 2 case. Appl. Statist 59, [10] Papa`roditis, E. and Politis, D. (2005). Bootstrap hypothesis testing in regression models, Statistics and Probability Letters, 74: [11] Quenouille, M. H. (1956). Notes on bias in estimation. Biometrika 43: [12] Xu, K. (2008). Bootstrapping autoregression under nonstationary volatility, Econometrics Journal, 11: 1-26.