Bootstrap Testing in Econometrics

Transcription

1 Presented May 29, 1999 at the CEA Annual Meeting Bootstrap Testing in Econometrics James G MacKinnon Queen s University at Kingston Introduction: Economists routinely compute test statistics of which the finitesample distributions are unknown and use them to reject, or not reject, whatever hypotheses are being tested Let ˆτ denote the realized value of such a test statistic For simplicity, and without loss of generality, assume that we wish to reject if ˆτ is sufficiently large In principle, ˆτ might be almost any sort of test statistic based on almost any sort of model and almost any sort of data set In practice, it is usually based on the LR, LM, or Wald principles and has a known asymptotic distribution Asymptotic Theory Approaches: Reject if ˆτ > C a α, where C a α is the asymptotic critical value for a test at level α; Compute asymptotic P value, or marginal significance level, p a (ˆτ), and reject if p a (ˆτ) < α These lead to the same decision, but the second approach usually preserves more of the information that ˆτ contains When asymptotic theory provides a poor approximation to the finite-sample distribution of τ, these approaches work very badly Copyright c 1999 James G MacKinnon 1

2 Examples: Asymptotic tests often work badly for tests in SUR and simultaneous equations systems; OPG form of the information matrix test and other LM tests; nonnested hypothesis tests The Bootstrap Approach: 1 Specify a way to generate bootstrap samples that resemble the real data while satisfying the null hypothesis Let ˆµ denote this bootstrap data-generating process, or bootstrap DGP 2 Using ˆµ, generate B bootstrap samples indexed by j From each of them, compute a bootstrap test statistic τ j 3 Using ˆτ and the τj, compute either a bootstrap critical value or a bootstrap P value: To estimate a bootstrap critical value Ĉ α for a test at level α, first sort the τj from largest to smallest Then calculate Ĉ α τ α(b+1), assuming that α(b + 1) is an integer For example, if α = 05 and B = 999, Ĉ 05 would be τ 50 To estimate a bootstrap P value, let ˆp (ˆτ) = 1 B B I(τj > ˆτ) j=1 For example, if B = 999 and 73 of the τj then ˆp (ˆτ) = 73/999 = Reject the null hypothesis if ˆτ > C α or if ˆp (ˆτ) < α are greater than ˆτ, Copyright c 1999 James G MacKinnon 2

3 Ideal and Feasible Bootstrap Tests: When B is finite, the feasible bootstrap P value ˆp (ˆτ) will depend on the random numbers used to generate the bootstrap samples As B, a law of large numbers implies that it will tend to the ideal bootstrap P value p (ˆτ) Pr ˆµ (τ ˆτ) (1) However, neither of these is, in general, equal to the true P value: p(ˆτ) Pr µ (τ ˆτ) (2) Questions: Will bootstrap tests ever give the correct answer if B is infinite? Will they ever give the correct answer if B is finite? How should we choose the number of bootstrap samples, B? Does bootstrap testing involve any loss of power relative to asymptotic testing? Does it make any difference whether we compute bootstrap P values or bootstrap critical values? Will bootstrap tests be more accurate than the asymptotic tests on which they are based? How should we specify the bootstrap DGP, ˆµ, and generate the bootstrap samples? When can we expect bootstrap tests to work well? When can we expect them to work badly? Copyright c 1999 James G MacKinnon 3

4 Pivotal and Nonpivotal Tests: A test statistic τ is pivotal if the distribution of τ does not depend on anything that is not observed by the econometrician In particular, it does not depend on unknown parameters If a test has a known finite-sample distribution, then it is pivotal But this is not necessary for a test to be pivotal For example, in the context of a regression model like y = Xβ + u, u N(0, σ 2 I), (3) where X can be treated as fixed, many test statistics have distributions that depend on X but not on β or σ 2 These test statistics are pivotal A simple example is the Durbin-Watson statistic for testing against first-order serial correlation: d = n t=2 (û t û t 1 ) 2 n t=1 û2 t (4) The û t are elements of the vector M X y Under the null hypothesis, M X y = M X u = σm X ε, where ε N(0, I) (5) Since the factors of σ cancel out in (4), d does not depend on any unknown parameters Therefore, it is pivotal Copyright c 1999 James G MacKinnon 4

5 Fundamental Result: Ideal bootstrap tests will yield valid inferences whenever τ is pivotal, because p (ˆτ) = p(ˆτ) Proof: If τ is pivotal, its distribution under ˆµ must be the same as its distribution under the true DGP, µ Therefore, Pr ˆµ (τ ˆτ) = Pr µ (τ ˆτ) (6) Thus, in this case, the ideal bootstrap test works perfectly under the null hypothesis Examples: In the context of the linear regression model with IID normal errors and fixed regressors, the following are pivotal: tests for autoregressive and/or moving-average errors of any order that just depend on the residuals and X; tests for heteroskedasticity, including tests for ARCH errors; tests for skewness and kurtosis that depend on the residuals; information matrix tests For pivotal tests of this type, the bootstrap samples are very simple to generate Since all test statistics are functions of M X ε, we just have to generate ε from the DGP There is no need to compute u or y at all ε N(0, I) (7) Warning: The IID normal assumption is critical, although any other known distribution would do equally well So is the assumption that there are no lagged dependent variables and no other regressors that may depend on lagged values of the dependent variable Copyright c 1999 James G MacKinnon 5

6 Specifying the Bootstrap DGP: This is the only part of bootstrap testing that can be difficult We have to specify a DGP that generates bootstrap samples with the same characteristics that the real sample would have had if the null hypothesis were true For the linear regression model (3), the parametric bootstrap works as follows: Estimate the model by OLS under the null hypothesis to obtain ˆβ and s Generate the bootstrap samples from the DGP y = Xˆβ + u, u N(0, s 2 I) (8) Unless the model is dynamic, X will be the same for all bootstrap samples If the model is dynamic, generate the yt recursively For an AR(1) model with a constant term, the DGP would be y t = ˆβ 1 + ˆβ 2 y t 1 + u t, u t NID(0, s 2 ) (9) Either sample from the unconditional distribution to obtain the presample values of yt or use the observed presample values of y t It is critical to impose the null hypothesis For example, if β = [β 1 β 2 ] and the null hypothesis is that β 2 = 0, we actually estimate the model y = X 1 β 1 + u (10) and use ˆβ = [ ˆβ 1 0] to generate the bootstrap samples Copyright c 1999 James G MacKinnon 6

7 We often do not want to assume that the error terms are normally distributed, but we are willing to assume that they are IID If so, we can use a semiparametric bootstrap, also called a nonparametric bootstrap, of which there are several varieties The simplest method is to bootstrap the residuals Suppose the original estimation yields residuals û 1, û 2,, û n Then we obtain bootstrap error terms by resampling with replacement from the residuals Each element of each vector of bootstrap errors is one of the original residuals, chosen at random with probability 1/n This may seem bizarre, but it generally works well, at least for regression models Even though the bootstrap errors come from a discrete distribution, the discreteness largely averages out when the test statistic is computed To ensure that the u t have variance s 2, it is better to resample from the rescaled residuals ( ) 1/2 n û t (11) n k We could also resample from the leverage-adjusted residuals ( ) ( 1/2 n û t n 1 (1 h t ) 1 1/2 n n s=1 û s (1 h s ) 1/2 ) (12) More sophisticated methods, such as applying a kernel smoother to the residuals and then resampling from the estimated distribution, are also available and may work better when the sample size is small When the IID assumption is inappropriate we can use the wild bootstrap or resort to bootstrapping pairs These let us handle heteroskedasticity, but the latter makes bootstrap testing tricky For serially dependent data, we can use the moving block bootstrap Copyright c 1999 James G MacKinnon 7

8 How Should We Choose B?: For a test at level α, always choose B so that α(b + 1) is an integer Thus, for α = 05 and α = 01, the smallest possible values of B are 19 and 99 A better choice is 999 Imagine that we sort all B + 1 test statistics, ˆτ and the B bootstrap ones, from largest to smallest, and reject the null if the rank of ˆτ is less than or equal to α(b + 1) If τ is pivotal, this procedure is called a Monte Carlo test, and it is exact The rank of ˆτ can have B + 1 possible values, all equally likely under the null, because ˆτ and the τj come from the same distribution Since α(b + 1) is an integer, the null will be rejected in exactly α(b + 1) of the B + 1 possible cases, that is, with probability equal to α This is equivalent to rejecting when ˆp < α, because we will have ˆp < α whenever the rank of ˆτ in the sorted list is less than α(b + 1) It is also equivalent to estimating the critical value Ĉ α as τ α(b+1) and rejecting if ˆτ > Ĉ α Example: Suppose that B = 99 and α = 05 If ˆτ is ranked fifth, then ˆp = 4/99 = 0404, and we will reject the null If ˆτ is ranked sixth, then ˆp = 5/99 = 0505, and we will not reject Why not choose B = 99? Outcome of test may depend on random numbers used to generate bootstrap samples This problem goes away as B There will be some loss of power relative to using B = Increasing the number of bootstrap samples from B to B +1/α, when α(b + 1) is an integer, will increase power Power loss depends on the shape of the size-power curve, which graphs the power of a test against its level Copyright c 1999 James G MacKinnon 8

9 Power Loss from Bootstrapping: F test (equivalently, two-tailed t test) for the null hypothesis that γ = 0 in the model y t = γ + u t, u t N(0, 1), t = 1,, 4 (13) Ideal bootstrap P value p was obtained from F (1, 3) distribution Feasible bootstrap P value ˆp was obtained by bootstrapping using various values of B There were one million replications Power γ = 30 γ = 20 γ = 10 γ = Size Figure 1 Size-power curves for B = Copyright c 1999 James G MacKinnon 9

10 Power Loss B = B = B = B = Size Figure 2 Power loss from bootstrapping, γ = 1 Power Loss B = B = B = B 000 = 399 Size Figure 3 Power loss from bootstrapping, γ = 2 Copyright c 1999 James G MacKinnon 10

11 Power Loss B = B = B = B = Size Figure 4 Power loss from bootstrapping, γ = 3 Conclusions: Power loss tends to be small when the size-power curve is close to the 45 line Power loss is extremely small when power is extremely high Power loss can be severe for tests at conventional significance levels, especially the 01 level, when the tests are not extremely powerful For tests at the 05 level, use at least B = 399 and preferably B = 999 For tests at the 01 level, use at least B = 1499 and preferably B = 3999 Copyright c 1999 James G MacKinnon 11

12 Choosing B by Pretesting If B is not fixed in advance, we may be able to get away with quite a small number of bootstrap samples Suppose that ˆτ is substantially more extreme than any of just 19 realizations of τj Then we can reject at the 05 level, and we can do so with reasonable confidence If ˆp is sufficiently far from α, even if B is small, we can be confident that more bootstraps will not change our conclusion This suggests choosing B endogenously, as follows: 1 Choose B min, the initial number of bootstrap samples (say, 99), B max, the maximum number of bootstrap samples (say, 12,799) and β, the level for the pretest (say, 001) Initially, calculate τ j for B min bootstrap samples, and set B = B min and B = B min 2 Compute ˆp (ˆτ) based on B bootstrap samples Depending on whether ˆp (ˆτ) < α or ˆp (ˆτ) > α, test either the hypothesis that p (ˆτ) α or the hypothesis that p (ˆτ) α at level β This may be done using the normal approximation to the binomial distribution If ˆp (ˆτ) < α and the hypothesis that p (ˆτ) α is rejected, or if ˆp (ˆτ) > α and the hypothesis that p (ˆτ) α is rejected, stop 3 If the null is not rejected, set B = 2B + 1 If B > B max, stop Otherwise, calculate τj for a further B + 1 bootstrap samples and set B = B Then return to step 2 This procedure automatically tends to use larger values of B when p is near α, which is what we want It seems to work well: Using a fixed value of B equal to the average value eventually chosen leads to lower power and more conflicts Copyright c 1999 James G MacKinnon 12

13 The Size Distortion of Bootstrap Tests: Although some interesting test statistics are pivotal, many are not When τ is not pivotal, bootstrapping will not yield an exact test Since ˆµ µ, the τj will not have the same distribution as ˆτ This implies that the bootstrap P value p (ˆτ) will not equal the true P value p(ˆτ), even if B = Define π τ as the asymptotic P value associated with τ For example, if τ a χ 2 (1), then π τ will be equal to 1 χ 2 1(ˆτ) Then the rejection probability function, or RPF, is defined as R(α, µ) Pr µ (π τ α) (14) Evidently, R(α, µ) depends on α and on the DGP µ For exact test, RPF is equal to α everywhere For pivotal test, RPF is flat, but generally not equal to α For nonpivotal test, RPF is not flat For asymptotically pivotal test, RPF becomes flat as n Typically, the RPF approaches a horizontal line at a rate proportional to n 1/2 or n 1 Copyright c 1999 James G MacKinnon 13

14 R(α, θ) θ 1 θ 2 θ 3 θ 4 θ Figure 5 A rejection probability function Implications: If a P value function is (locally) flat, the bootstrap test will work perfectly How well or badly the asymptotic test performs has no effect on the performance of the bootstrap test The slope of the RPF matters only if the estimates of the parameters that determine ˆµ are biased The curvature of the RPF always matters Copyright c 1999 James G MacKinnon 14

15 A bootstrap test will usually perform better if µ is estimated more efficiently and/or with less bias It is therefore best to use restricted estimates rather than unrestricted estimates that have been modified in order to satisfy the null hypothesis For asymptotically pivotal tests, there are two sources of improvement as n increases First, the RPF becomes flatter, which improves the performance of bootstrap and asymptotic tests equally Second, ˆµ µ, which further improves the performance of the bootstrap test The details of how ˆµ are constructed may matter If a test statistic is asymptotically pivotal, a bootstrap test based on it will perform better than the corresponding asymptotic test, in the sense that it will commit errors of lower order in n One-tailed tests: asymptotic O(n 1/2 ) bootstrap O(n 1 ) Two-tailed tests: asymptotic O(n 1 ) bootstrap O(n 2 ) In some circumstances, bootstrap tests may do even better When a test statistic is not asymptotically pivotal, a bootstrap test based on it will generally perform no better than most asymptotic tests It will make errors that are either O(n 1/2 ) or O(n 1 ) Typically, n 2 is very much smaller than n 1 Therefore, we can expect bootstrap tests to work very well in many cases Copyright c 1999 James G MacKinnon 15

16 The Power of Bootstrap Tests: It is natural to worry that correcting test size by bootstrapping may have harmful effects on test power Luckily, such worries are misplaced For a pivotal test, there can be no power difference at all, since the bootstrap test is equivalent to an asymptotic test that has been size-corrected In general, the bootstrap and asymptotic tests will have power that differs, on a size-corrected basis, only at O(n k/2 ) In most cases, k = 2 or k = 3 This discrepancy is of the same order as the error in the bootstrap test under the null Any power difference that may occur is due solely to the fact that the difference between the bootstrap critical value and the true critical value may differ under the DGPs that correspond to the null and alternative hypotheses If this difference is not small to begin with, the asymptotic test will be seriously inaccurate There are many ways to size-correct an asymptotic test that is not pivotal If a Monte Carlo experiment shows apparent loss, or gain, of power due to bootstrapping, it is probably because of the way the asymptotic test was size-corrected Even if a size-corrected asymptotic test appears to be a little more powerful than a bootstrap test based on it, there is no advantage to using the former if it overrejects, or underrejects, much more severely than the bootstrap test Copyright c 1999 James G MacKinnon 16

17 Example: Testing for AR Errors: Tests for autoregressive errors in a linear regression model with a lagged dependent variable are not pivotal Consider the model y t = X t β + δy t 1 + u t, u t = ρu t l + ε t, ε t NID(0, σ 2 ) (15) A popular test statistic is the t statistic for ρ = 0 in the regression y t = X t β + δy t 1 + ρû t l + residual, (16) where û t is the t th residual from OLS estimation of (15) These tests are not at all expensive to bootstrap Less than 1/5 of a second for 999 bootstrap samples on a Pentium II 450 for n = 100 with 10 regressors; even cheaper if no lagged dependent variable Monte Carlo Experiments: 5 regressors, one of them a constant term, others generated by AR(1) processes with parameters of 05 or 09 β = [ ], δ = 08 or δ = 095, σ = 1 n = 8, 9, 10, 11, 12, 14, 16, 18, 20, 25,, 55, 60 B = ,000 replications Rejection frequencies for ordinary t or F test compared with parametric bootstrap rejection frequencies at 5% level Copyright c 1999 James G MacKinnon 17

18 t test bootstrap n Figure 6 Rejection frequencies for bootstrap and t tests, δ = t test bootstrap n Figure 7 Rejection frequencies for bootstrap and t tests, δ = 095 Copyright c 1999 James G MacKinnon 18

19 Example: Nonnested Hypothesis Tests: Competing models: H 1 : y = Xβ + u, u N(0, σ 2 I), and H 2 : y = Zγ + v, v N(0, σ 2 I), (17) where y, u, and v are n 1, X and Z are n k and n l, β is k 1, and γ is l 1 We wish to test H 1 J test statistic is ordinary t statistic for α = 0 in the regression y = Xb + αp Z y + residuals, (18) where P Z = Z(Z Z) 1 Z Thus P Z y is the vector of fitted values from OLS estimation of the H 2 model The J statistic for testing H 1 can be written as J = y P Z M X y, (19) ś(y P Z M X P Z y) 1/2 where P X X(X X) 1 X and M X I P X If the error terms are NID, the finite-sample distribution of J can be shown to depend on X, Z, and the parameters θ β/σ of H 1 The distribution of J is nonpivotal only because it depends on θ Large values of θ imply that the part of the H 1 model that is not also part of the H 2 model has substantial explanatory power relative to the size of the error terms Small values imply the opposite The J test appears to be extremely nonpivotal, at least near θ = 0; see Figure 8 It is much better behaved if at least one of the θ i is not allowed to approach 0; see Figure 9 Copyright c 1999 James G MacKinnon 19

20 N = N = 20 N = N = 80 N = θ 1, θ Figure 8 Rejection frequencies for J Test, both parameters varying θ 1 = 1 N = 10 N = 20 N = 40 N = 80 N = Figure 9 Rejection frequencies for J test, one parameter varying θ 2 Copyright c 1999 James G MacKinnon 20

21 Bootstrapping the J test Bootstrapping the J test is very easy Figures 10 and 11 deal with cases where the asymptotic J test works very badly and the bootstrap J test might be expected to work poorly because the RPF is far from flat N is n minus the number of regressors that are common to both X and Z There is a large number of nonoverlapping regressors, which makes both asymptotic and bootstrap tests perform poorly Figure 10: X has 2 regressors that are not in Z, and Z has 6 regressors that are not in X Figure 12: X has 5 regressors that are not in Z, and Z has 9 regressors that are not in X Even in these extreme cases, where the asymptotic tests perform dreadfully (see Figures 11 and 13), the bootstrap J test performs remarkably well Why do the bootstrap J tests work so well? There is little bias in the estimation of θ, especially when θ is small, which is where the RPF is most nonlinear The standard errors of ˆθ are small, especially when θ is small Therefore, the bootstrap distribution is rarely being evaluated very far from θ 0 When the RPF is very steep, it is almost linear Therefore, since ˆθ is almost unbiased, errors tend to cancel out Most experiments have dealt with cases where asymptotic J tests work very badly, and where theory suggests that bootstrap tests should work relatively badly There is thus strong reason to believe that bootstrap J will work extremely well in practice Copyright c 1999 James G MacKinnon 21

22 θ i = 00 θ i = 01 θ i = 02 θ i = 05 θ i = 10 θ i = N Figure 10 Rejection Frequencies for Bootstrap Tests: Case θ i = 02 θ i = 10 θ i = 00 θ i = 01 θ i = 05 θ i = Figure 11 Rejection Frequencies for Asymptotic Tests: Case 1 N Copyright c 1999 James G MacKinnon 22

23 0090 θ i = 00 θ i = θ i = 02 θ i = 05 θ i = θ i = N Figure 12 Rejection Frequencies for Bootstrap Tests: Case θ i = θ i = θ 060 i = θ i = 05 θ i = θ i = N Figure 13 Rejection Frequencies for Asymptotic Tests: Case 2 Copyright c 1999 James G MacKinnon 23

24 Obstacles to Bootstrap Testing: Bootstrap testing is conceptually easy whenever the null hypothesis is a fully specified, parametric model We simply use estimates of the model under the null to generate the bootstrap samples However, even in this case, there can be difficulties If nonlinear estimation is needed for every bootstrap sample, the nonlinear optimization program needs to be fast and must operate reliably without human intervention Sometimes, nonlinear estimation can be replaced by a small, fixed number of steps, similar to one-step efficient estimation But it must be verified that this works For time-series applications, it is essential to ensure that the bootstrap DGP is stationary We can easily relax the assumption that the error terms follow a known distribution if we are willing to assume that they are IID Then we can resample from rescaled residuals or use a more sophisticated semiparametric procedure When the error terms are heteroskedastic, things get harder If the conditional variance only depends on a fixed matrix X, the wild bootstrap, in which the error terms for each observation are drawn from a different distribution, may work With autoregressive conditional heteroskedasticity, it appears to be necessary to estimate a parametric GARCH model and use it to generate the data The innovations may either be drawn from the normal distribution, or some other distribution, or resampled in the same ways as regression model error terms Copyright c 1999 James G MacKinnon 24

25 The bootstrap DGP must not treat as fixed variables which are really determined endogenously Therefore, simultaneous equations models and dynamic models both require care 1 Consider the model y = X 1 β + Y 1 γ + u (20) where Y 1 is a matrix of endogenous variables, and we wish to test some restriction on β or γ Simply obtaining 2SLS estimates of (20) under the null does not allow us to generate bootstrap samples We also need to estimate the reduced form Y 1 = XΠ + V, (21) and we will need the estimated covariance matrix 1 n [û ˆV ] [û ˆV ] (22) in order to implement a parametric bootstrap For a semiparametric bootstrap, we could resample from the rows of [û ˆV ] There is more than one sensible way to estimate (20) and (21) under the null hypothesis, and there is consequently more than one valid way to generate the bootstrap samples 2 Consider the dynamic model y t = β 1 + β 2 y t 1 + β 3 z t + u t (23) z t = X t γ + δy t 1 + v t (24) Even though OLS estimation of (23) alone is asymptotically valid when the u t are serially uncorrelated, we need to estimate both equations and generate both y t and z t dynamically when we form bootstrap samples Copyright c 1999 James G MacKinnon 25

26 Final Remarks: In many cases, it is no longer necessary to rely on asymptotic tests that can be highly inaccurate in finite samples Bootstrap tests will often allow us to make much more accurate inferences Parametric bootstrap tests will work perfectly whenever the test statistic is pivotal Parametric bootstrap tests will work almost perfectly for many cases in which the test statistic is asymptotically pivotal Parametric bootstrap tests will usually yield more accurate inferences than asymptotic tests, except perhaps for very small sample sizes Nonparametric bootstrap tests will also yield more accurate inferences than asymptotic tests in many cases, provided the sample size is not too small If the bootstrap DGP does not resemble the true DGP, and the test statistic is severely nonpivotal, bootstrap tests may not perform well The number of bootstrap samples, B, should be chosen with care It can be chosen using a pretest procedure Bootstrap testing need not be computationally expensive, if appropriate software is used Bootstrapping should have little effect on size-adjusted power, if it is measured correctly Copyright c 1999 James G MacKinnon 26

27 Books: References Davison, A C and D V Hinkley (1997) Bootstrap Methods and Their Application, Cambridge, Cambridge University Press Hall, P (1992) The Bootstrap and Edgeworth Expansion, New York, Springer-Verlag Papers: Beran, R (1988) Prepivoting test statistics: a bootstrap view of asymptotic refinements, Journal of the American Statistical Association, 83, Davidson, R and J G MacKinnon, The size distortion of bootstrap tests, Econometric Theory, 15, 1999, forthcoming Davidson, R and J G MacKinnon, Bootstrap testing in nonlinear models, International Economic Review, 40, 1999, Davidson, R and J G MacKinnon, Bootstrap tests: how many bootstraps? Econometric Reviews, 18, 1999, forthcoming Dufour, J-M and J F Kiviet (1998) Exact inference methods for first-order autoregressive distributed lag models, Econometrica, 66, DiCiccio, T J and D Efron (1996) Bootstrap confidence intervals, Statistical Science, 11, Freedman, D A and S C Peters (1984) Bootstrapping an econometric model: some empirical results, Journal of Business and Economic Statistics, 2, Copyright c 1999 James G MacKinnon 27

28 Godfrey, L G (1998) Tests of non-nested regression models: Some results on small sample behaviour and the bootstrap, Journal of Econometrics, 84, Hall, P and J L Horowitz (1996) Bootstrap critical values for tests based on generalized-method-of-moments estimators, Econometrica, 64, Horowitz, J L (1994) Bootstrap-based critical values for the information matrix test, Journal of Econometrics, 61, Horowitz, J L (1997) Bootstrap methods in econometrics: theory and numerical performance, in Advances in Economics and Econometrics: Theory and Applications, Vol 3, ed D M Kreps and K F Wallis, Cambridge, Cambridge Univ Press Horowitz, J L (1998) Bootstrap Methods for Median Regression Models, Econometrica, 66, Jöckel, K-H (1986) Finite sample properties and asymptotic efficiency of Monte Carlo tests, Annals of Statistics, 14, Li, H and G S Maddala (1996) Bootstrapping time series models, Econometric Reviews, 15, Li, Q and S Wang (1998) A simple consistent bootstrap test for a parametric regression function, Journal of Econometrics, 87, Liu, R Y (1988) Bootstrap procedures under some non-iid models, Annals of Statistics, 16, Mammen, E (1993) Bootstrap and wild bootstrap for high dimensional linear models, Annals of Statistics, 21, Nankervis, J C and N E Savin (1996) The level and power of the bootstrap t test in the AR(1) model with trend, Journal of Business and Economic Statistics, 14, Copyright c 1999 James G MacKinnon 28