Least Squares Bias in Time Series with Moderate. Deviations from a Unit Root

Transcription

1 Least Squares Bias in ime Series with Moderate Deviations from a Unit Root Marian Z. Stoykov University of Essex March 7 Abstract his paper derives the approximate bias of the least squares estimator of the autoregressive coefficient in discrete autoregressive time series where the autoregressive coefficient is given by α = + c/k, with k being a deterministic sequence increasing to infinity at a rate slower than, such that k = o ) as. he cases in which c <, c = and c > are considered, corresponding to stationary, non-stationary and explosive series. Keywords: Autoregressive bias; moment generating function. Address for correspondence: Marian Z. Stoykov, Essex Business School, University of Essex, Wivenhoe Park, Colchester, Essex CO4 3SQ, England; mzstoy@essex.ac.uk

2 Introduction Economics and finance researchers and practitioners rely heavily on autoregressive time series models. he assumption that the value of an asset, or GDP, inflation, etc., at time period t depends on the value of the same variable at the previous period, t, seems quite plausible to make. As such, to make any valid inference, the properties of parameter estimators in stochastic difference equation models need to be well understood. o try and make our task simpler, in most cases we rely on asymptotic theory as an approximation to finite sample distributions as asymptotic distributions usually have very simple forms. For example, by applying the central limit theorem to the ordinary least squares OLS) estimator of the autoregressive coefficient in stationary time series, it can be shown that under a particular set of assumptions the former is asymptotically normally distributed with mean zero and a well defined variance. From there, it is straightforward to construct confidence intervals and utilise them for inference. However, it does not have to be the case that any asymptotic distribution is shared by its finite sample counterpart. For example, the exact maximum likelihood MLE) and OLS estimators share the same asymptotic distribution but differ in terms of their finite sample behaviour as they treat the initial condition differently the initial condition is asymptotically negligible). hus, having information on their asymptotic behaviour only is not enough as a guide on which estimator is to be preferred over the other when applied to finite samples settings. In addition, asymptotic theory relies on having unlimited samples, something too luxurious to have in practice. It should also be noted that the OLS estimator has different properties when it is applied to non-stationary and explosive series. All of the above mentioned becomes very important, especially in macroeconomic settings, since observations of GDP, inflation, etc. are very limited, as most of the macroeconomic variables are usually available quarterly. hus, it would be of help to know how estimators perform in finite samples. One of the main features of the OLS estimator of the autoregressive parameter is that it is downward negatively) biased for any finite sample. However, the bias vanishes asymptotically. his result holds regardless of whether the data generating process produces stationary, nonstationary or explosive series. his characteristic of the OLS method has been demonstrated both theoretically and via simulations, with many authors having contributed to the topic. In terms of stationary series, Hurwicz 95) and White 96) derived the result of the bias by

3 means of series expansions up to order O 4), where is the sample length. For the nonstationary case, Phillips 987) provides an expansion up to O ) order. However, fewer authors have focused on the explosive side. Le Breton and Pham 989) derive the first order term of the bias explicitly and Phillips ) only derives the asymptotic order of the bias. he above mentioned papers deal with AR) processes with normally distributed errors. Shaman and Stine 988) extend the literature by considering AR models of higher order which are driven by normal errors and Bao 7) derives the bias for an AR) process where the errors are allowed to follow any distribution. hese setups take the autoregressive coefficient as fixed and as either smaller, equal or bigger than one. his means one would need to know a priori what the data generating process is. his, for example, could have applications for modelling stock prices as returns are stationary. However, it will be misleading to use the same results for near-integrated processes. As such, Phillips ), has considered the local to unit root cases where the autoregressive coefficient is given by α = + c/ for both c bigger and smaller than zero and by allowing for c and c. Recent development in the asymptotic theory of AR processes with an autoregressive coefficient given by α = + c/k, where k is a sequence which increases to infinity at a rate slower than, such that k = o ) as, has been made by Phillips and Magdalinos 7) hereafter, PM). he OLS estimator of the autoregressive parameter in this case of moderate deviations from a unit root follows an asymptotic distribution which is equivalent to the one that is obtained by considering a fixed autoregressive parameter. his framework was utilised by Phillips, Wu and Yu ) to test the NASDAQ index for explosive behaviour. Given those recent advancements in the literature it would be of interest to derive the bias of the OLS estimator for such processes. he paper is organised as follows: section summarises the main results and provides a discussion. Section 3 concludes, and the technical details are collected in the Appendix. For the bigger part of the paper, with the exception of the discussion in section, the lower script in α is dropped out for notational simplicity; the symbol will be used for summations running from t = to ; W r) will be used to denote a Wiener process on C[, ], the space of continuous real-valued functions on the unit interval and denotes convergence in distribution. 3

4 Main Results Suppose x t is given by the following stochastic difference equation x t = α x t + u t, α = + c k for t =,...,, ) where u t is identically and independently distributed N, σ ) and k is a sequence that increases to infinity, such that k = o ) as. he distribution of x = x,..., x ) in ) is uniquely determined by specifying an initial condition for the process. he density function of x for a constant initial condition, x = γ is given by fx) = πσ ) / xt αx t ) exp { } σ. If the initial condition is specified not as a constant but as a random variable given by x σ ) N, α the density of x becomes f x) = α ) / { πσ ) / exp α )x + x t αx t ) } σ. he model which this paper will consider is the former. Unfortunately, when α > the exact MLE for the latter is inconsistent due to the specification of the initial condition. Hamilton 994, pp. 8-3) provides an excellent treatment on the subject. In particular, we will take γ =. he MLE for the constant initial condition, which coincides with the OLS estimator, is given by ˆα = xt x t x t. o derive the bias, we will make use of the joint moment generating function MGF) of the numerator and denominator of the result for ˆα. We may assume σ = as ˆα is independent of σ. Following the procedure of White, define U = x t x t and V = x t such that their 4

5 joint MGF is given by E expuu + V v) = mu, v) = expuu + V v)fx)dx { = π) / exp [ α + u)u + + α v)v + x ] } dx = π) / exp { } xdx dx = D u, v) /, ) { where the second line follows from fx) = π) / exp V + x αu + α V )} for σ = and pv) qu) qu) pv) qu) qu) pv) qu) D u, v)=......,..... qu) pv) qu) qu) with pv) = + α v and qu) = α + u). he last equality in ) is a result which can be found in Cramér 946, pp. 8-). Shenton and Johnson 965) showed that Eˆα α) = α D v) / dv, 3) where D v) is the determinant of the matrix evaluated at u = and v. Now the right hand side of 3) can be utilised to derive the bias the details are given in the appendix). he results are summarised in the following theorem. heorem. For the model considered in ), with x =, as, the bias of the autoregressive coefficient Eˆα α ) has the following asymptotic expansion α Eˆα α ) =.784 { ))} k [ + O max, k + O )], c <, π/ c 3/ / k 3/ [ + O )], c =, + α { ))} k [ + O max, k + O )], c >. 5

6 Remark. he bias is negative for all values of c. Starting with c <, which corresponds to α <, the first term of the expansion is α /. For a fixed α the first order term of the expansion is α/ e.g. White). hus, up to the first order term the two results are equivalent. However, the results differ in higher order terms. he second order term for the fixed case is 4α/ e.g. White), which is different from the result for c < due to the presence of the + O max k /, k )) term. Furthermore, it is not clear what the second order term of the expansion is until one specifies k, as the result in heorem is very general. One consistent with the definition of k parameterisation would be to set k = δ, with δ, ). In this case, the cut-off point is δ = /. hus, for δ, /], max k /, k ) = k = δ and for δ [/, ), max k /, k ) = k / = δ. Remark. For c =, corresponding to α =, a unit root process, the constant is a well known result e.g. anaka 996), p4). It is the expectation of the functional / W r)dw r) W r) dr, which is the leading term of the asymptotic bias for the unit root process see Phillips 987)). Remark 3. π / c 3/ / k 3/ For c >, which corresponds to α >, the first term of the expansion is +) α. From Le Breton and Pham, the respective term from the expansion for a fixed α is / π / α ) 3/ / α +). Note that α = + c/k ) = )) c/k + O k. hus, for the first order term of the expansion the two results are equivalent. he result of Le Breton and Pham does not provide the second order term of the expansion for the fixed case and as such comparison between the two cannot be made. hus, the present paper sheds some light into the higher order terms of the expansion on the explosive side. aking a closer look into the result, it is perhaps surprising that the second-order term is not of order O α ) higher in magnitude than the first. One would have expected that the order would be a multiple of α as the consecutive terms in the stationary and non-stationary cases are a multiple of. Remark 4. his formulation, namely α = + c/k, suffers from the same problem as the result of Le Breton and Pham. heir analytical result is discontinuous in that the limits of their bias expressions α and α are different. his is also the case in the analytical result of heorem from the present paper, namely, the limits c and c are not identical. his is not surprising as the result from this paper and the of Le Breton and Pham are asymptotically equivalent up to the first term. he solid line in Figure depicts the result with = 4, 6

7 . Bias: Analytical Solution simulated analytical α=+c/ / Figure : Bias: Analytical Solution. k = and c [, 3], where the lower limit of c was chosen as to match with α = + c/ =. he lonely dot is the result for c =. he graph also shows the simulated OLS bias for the autoregressive coefficient of ) with x = and σ = for comparison. he number of observations is = 4 and the graph has been smoothed. It can be seen that for negative values of c or high positive values of the parameter the analytical solution provides a good approximation. Remark 5. PM provide an interesting discussion on the comparison between a fixed α = +c and the moderate deviations from a unit root given by α = + c/k frameworks, where k was defined in remark. heir aim is to find out whether taking the limits δ, produces the correct rates of convergence and asymptotic distributions. hey show this to be the case only for the boundary limit δ. When k = δ from heorem 3. of PM, as, we have /+δ/ ˆα α ) N, c) for c <, 4) δ α ˆα α ) C c for c >, 5) where C is the standard Cauchy random variable. In comparison, it is well-known that for the 7

8 model considered in ) and a fixed α / ˆα α) N, α ) for α = + c <, 6) α α ˆα α) C for α = + c >, 7) where the result in 7) is due to White 958). From 4) and 6) there is a discrepancy between the terms c and α = c c, and from 5) and 7) between c and α = c + c. Continuity can be achieved by substituting c by c + c / δ without affecting the asymptotic distributions and moments. his argument does not apply for the case δ, as discussed by PM. In the same fashion, it would be interesting to check whether the same arguments hold for the bias as well. From heorem 3. of Le Breton and Pham, as Eˆα α) converges to α for α = + c < 8) / α Eˆα α) converges to / π / α α ) 3/ for α = + c >. 9) For c <, heorem and heorem 3. from Le Breton and Pham produce the same result as δ without any adjustment. However, for c > heorem and 9) involve an α = c+c term. hus, continuity can be achieved by the substitution proposed by PM. his result follows immediately by substituting k with at the beginning of the integral expressions from the appendix and is stated as a corollary to heorem. Corollary. For model ) with x = and a fixed α = + c, as, the bias of the autoregressive coefficient Eˆα α) has the following asymptotic expansion Eˆα α) = + c).784 [ + O )], c <, [ + O )], c =, π/ c + c ) 3/ / [ + O )] / α +, c >. here is nothing surprising regarding this result given the discussion around heorem. However, it shows the magnitude of the second order term on the explosive side for a fixed autoregressive coefficient, which is new to the literature. Lastly, as in the discussion of PM, taking the limit 8

9 δ does not produce the same asymptotic bias. hese results are interesting. One would have perhaps expected that there should be no need for adjustment for the asymptotic bias, only for the asymptotic variance, in view of the fact that only the variances of the asymptotic distributions depend on either α or α for the stationary or explosive cases respectively. Remark 6. One way to try and utilise the result in heorem is to get an estimate of the bias and subtract it from the OLS estimator. MacKinnon and Smith 998) propose a procedure to reduce bias in autoregressive series with an intercept, which utilises a result similar to the one in heorem for a fixed α parameter. Another estimator which employs a similar idea is the jackknife estimator. Although the jackknife utilises sub-samples, it requires an asymptotic expression for the bias of the original estimator for the full-sample and a number of sub-samples such as the one provided in heorem. Chambers 3) and Chambers and Kyriacou 3) show how bias reduction can be achieved in stationary and non-stationary autoregressive series respectively. heorem shows that the analysis of Chambers can directly be translated to mildly stationary series. he author of the present paper has attempted to construct the jackknife estimator for mildly explosive autoregressive series but was unsuccessful due to the severe righthand discontinuity, which was subsequently derived in this paper. As such, heorem depicts why it would be impossible to construct the jackknife for mildly explosive series for positive values of c which are close to zero. he issue can be tackled by either substituting what should be the correct explosive weights by the ones applied to stationary series Kaufmann and Kruse 5) provide an interesting comparison between a number of estimators) or by considering local to unit root alternatives Chambers and Kyriacou 6) and Stoykov 6)), where the approximate bias function has been shown to be continuous at c = Phillips )). 3 Conclusion his paper has had the aim to derive the approximate bias of the autoregressive parameter in the discrete time autoregressive process of order one. he autoregressive coefficient is given by α = + c/k, where c is a constant and k is a sequence that is increasing to infinity at a slower rate than, the sample size, such that k = o ) as. he bias is shown to be negative for the three cases considered, namely c <, c = and c >, corresponding to stationary, non-stationary and explosive series respectively. he result is also discontinuous in 9

10 that the limits c and c are not identical in the final bias expression. Acknowledgements I am grateful to my supervisor Marcus Chambers for useful comments and suggestions on this work. I would also like to thank the Essex Business School for scholarship support. A Appendix o evaluate the determinant in 3) we note that it can be written in the form of a second order difference equation. Define κ p v) = + α + v such that D v) = κ D v) α D v), with initial conditions D v) = and D v) = κ α. Define the positive root of the characteristic equation as λ = κ + κ 4α ) A.) such that the solution to the homogeneous difference equation has the form ) α D v) = C λ + C. λ From the initial conditions the complete solution is given by ) λ α D v) = λ α λ + λ ) α λ α α λ. Phillips ) showed that D v) is positive for all v > meaning the integral in 3) is well-defined for all α. aking the derivative in 3) leads to Eˆα α) = D v) 3/ D v) dv. α Following Phillips ), define x = /λ. It follows that x = κ + κ 4α = κ κ 4α κ κ 4α ) = κ κ 4α α,

11 from which it follows that κ κ 4α = α x. A.) From A.) and A.) we have κ κ 4α = α x and κ + κ 4α = x, and by adding them together we get κ = /x + α x. From κ = + α + v we can solve for v as a function of x v = { } x + α x α = x) α x). x We write with derivative v = x) α x) x dv dx = α x ) x x) α x), x, ], α, = x x )α x ), x [, ), α >, x <, x, ], α, >, x [, ), α >. As pointed out by Phillips ), v = vx) is a monotonic transformation over the two domains specified and as such the variable of integration in 3) can be changed. Applying the change of variable yields λ α λ α = x α α = x α x) x α x, λ α λ α = x α x) α x = x α x, leading to D v) = α x α x x + = x α x α x α x + x)α x α x )x = α x + x )α x α x )x = A x; α) α x, for α, )x B x; α) α x, for α >, )x A.3)

12 where A x; α) = α x + x)α x and B x; α) = α x + x )α x. hus, for α, by changing the variable of integration from v to x and by taking the derivative in 3) we have Eˆα α) = α = α = α D v) / dq [ ] A x; α) / [ α x α x )x x x 5 α x ) 3/ A x; α) / dx. ] dx A.4) Evaluating the derivative in A.4) gives α = 3 x 5 α x ) 3/ A x; α) / dx x 5 x 5 α x ) / αx )A x; α) / dx α x ) 3/ A x; α) 3/ α A x; α)dx. A.5) he derivative from the last line in A.5) is α A x; α) = α x + x)α x ) α = αx + x)α x. A.6) Combining A.4), A.5) and A.6), for c <, corresponding to α <, we have Eˆα α) = 3α + α x x 3 α α x ) / A x; α) / dx α x ) 3/ A x; α) 3/ dx 5 7 x α x ) 3/ A x; α) 3/ x)dx. A.7) o evaluate the expectation in A.7), we start with the first integral. Setting y = x gives dy = )x dx = )y dx. Note that A x; α) = α x + O α ). By substituting α = + c/k in the third line of the following derivations and using the facts that y b b a = + a log y + O ), k = o ) and k α = o) as the integral

13 becomes = x = = = = = α x ) / A x; α) / dx y 3 4 { } 3 +3 α y / y 4 α y dy + y )yα ) + c k + c k + log y + O )) ) + c k + c k + log y + O ) ) + Oα ) log y y 3 c k c / k 4 log y c k c α k y log y dy [ + O )] k log y + 4c + c / y 3 k 4 dy [ + O )] k log y + 4c + c + k k α y log y [ { })] 4c + O max k, k / y 3 4 [ { })] 4c + O max k, k ))} y 3 k [ 4 dy { + O max, k + O )]. / dy [ + O )] { ))} k [ dy + O max, k + O )] he second integral can be dealt with in the same fashion = = x 3 α x ) 3/ A x; α) 3/ dx y 3 4 { k k log y + 4c + c k } 3/ dy [ + O )] log y + 4c + c k + k α y log y ))} y 3 k [ 4 dy { + O max, k + O )]. he third integral becomes exponentially small as α = o) as. By combining the three integrals, for c <, we have [ Eˆα α) = 3α = α { + O y 3 α 4 dy + max k, k ] y 3 k 4 dy { + O max, k ))} [ + O )]. ))} [ + O )] A.8) 3

14 For c =, corresponding to α =, the algebra reduces to the one found in Phillips ). From A.5) as we have Eˆα ) = 3 + x x x { } + x / dx + x { } + x 3/ dx + x { } + x 3/ x)dx. + x By setting y = x the first integral becomes x { } + x / dx = + x = / y 3 4 { + y + y } / dy [ + O )] y 3 4 { + y} / dy [ + O )]. he second and third integrals can be dealt in the same fashion. he second becomes x 3 { } + x 3/ dx = 3/ + x y 3/4 { + y} 3/ dy [ + O )], and the third is given by { } x 3/ x x)dx = 3/ + x 4 y 4 { + y} 3/ log ydy [ + O )]. Combining the three integrals yields Eˆα ) = 3 / + 3/ 3/ 4 y 3 4 { + y} / dy [ + O )] y 3/4 { + y} 3/ dy [ + O )] y 4 { + y} 3/ log ydy [ + O )]). Evaluating the expression numerically yields Eˆα ) = O ). A.9) 4

15 Lastly, consider the case in which c >, corresponding to α >. From A.3) we have D v) = α x + x )α x α x )x = B x; α) α x )x From 3) and A.3), we have that for c > Eˆα α) = α = α = α D v) / dq [ ] B x; α) / [ α x ] α x )x x dx x 5 α x ) 3/ B x; α) / dx. A.) Evaluating the derivative in A.) gives α = 3 x 5 α x ) 3/ B x; α) / dx x x 5 α x ) 3/ B x; α) / αx dx α x ) 3/ B x; α) 3/ α B x; α)dx. A.) he derivative from the last line in A.) is α B x; α) = αx + x )α x. A.) Combining A.), A.) and A.), for c > we have Eˆα α) = 3α α x x 3 α α x ) / B x; α) / dx α x ) 3/ B x; α) 3/ dx 5 7 x α x ) 3/ B x; α) 3/ x )dx. A.3) Note that B x : α) = α x + x )α x. herefore, the first and second integrals in A.3) become exponentially small due to the α = e c { )} k + O k term which explodes as. As a result, the bias for c > is determined by the third integral which, as, 5

16 becomes α α = ) α = ) α = ) = ) = α 5 7 x α y 4 y 4 y 4 ) / k 3/ = α c) 3/ ) / k 3/ = c3/ / k 3/ α + α x ) 3/ 3/ B x )dx ) y ) α y α y + y ) yα α y α α α y ) + y 4 + c k y ) y ) ) + c + k log y y log y ) c log y + k y log y c + O max y log y + c k k, k ) 3/ ))) 3/ 3/ ) y dy 3/ ) ) y dy [ + O )] log ydy [ + O )] log ydy [ + O )] 3/ { y 4 {y log y} 3/ k log ydy + O max y 4 {y log y} 3/ log ydy Finally, setting log y = w such that dy = e w dw yields { k + O max, k log ydy [ + O )] ))} [ + O )], k ))} [ + O )]. c3/ / k 3/ α + = c3/ / k 3/ = c3/ / k 3/ α + α + = π/ c 3/ / k 3/ α + { ))} y 4 {y log y} 3/ k [ log ydy + O max, k + O )] ))} [ + O )] { e 5 4 w {we w } 3/ wdw + O max { e 4 w w / k dw + O max { k + O max, k k, k, k ))} [ + O )]. ))} [ + O )] A.4) he three results of the bias from A.8), A.9) and A.4) are collected in the heorem. 6

17 References [] Bao, Y. 7) he approximate moments of the least squares estimator for the stationary autoregressive model under a general error distribution. Econometric heory 3, 3-. [] Chambers, M.J. 3) Jackknife estimation of stationary autoregressive models, Journal of Econometrics, 7, [3] Chambers, M.J. and M. Kyriacou 3) Jackknife estimation with a unit root, Statistics and Probability Letter, 83, [4] Chambers, M.J. and M. Kyriacou 6) Jackknife bias reduction in the presence of a nearunit root, Economics Discussion Paper, Department of Economics, University of Essex. [5] Cramér, H. 946) Mathematical methods of statistics. Princeton University Press. [6] Hamilton, J.D. 994) ime series analysis. Princeton University Press. [7] Hurwicz, L. 95) Least squares bias in time series. Statistical Inference in Dynamic Economic Models, ed.. C. Koopmans). Wiley, New York. [8] Le Breton, A. and D.. Pham 989) On the bias of the least squares estimator for the first order autoregressive process. Annals of the Institute of Statistical Mathematics 4, [9] MacKinnon, J.G. and A.A. Smith 998) Approximate bias correction in econometrics, Journal of Econometrics 85, 5-3. [] Phillips, P.C.B. 987) ime series regression with a unit root. Econometrica 55, [] Phillips, P.C.B. ) Folklore theorems, implicit maps and indirect inference. Econometrica 8, [] Phillips, P.C.B and. Magdalinos 7) Limit theory for moderate deviations from a unit root Journal of Econometrics 36, 5-3. [3] Pillips, P.C.B, W. Yangru and Y. Jun ) Explosive behaviour in the 99s NASDAQ: When did exuberance escalate asset values?. International Economic Review 5, -6. [4] Shaman, P. and R.A. Stine 988) he bias of autoregressive coefficient estimators. Journal of the American Statistical Association 83,

18 [5] Shenton, L.D. and W.L. Johnson 965) Moments of a serial correlation coefficient. Journal of the Royal Statistical Society, Series B 7, [6] Stoykov, M.Z. 6) Optimal jackknife estimation of local to unit root models, PhD hesis, Essex Business School, University of essex. [7] anaka, K. 996) ime series analysis: Nonstationary and noninvertible distribution theory. New York: John Wiley and Sons. [8] White, J.S. 958) he limiting distribution of the serial correlation coefficient in the explosive case. Annals of Mathematical Statistics 9, [9] White, J.S. 96) Asymptotic expansion for the mean and variance of the serial correlation coefficient. Biometrika 48,