A Bootstrap Approach for Generalized Autocontour Testing. Implications for VIX forecast densities

Transcription

1 A Bootstrap Approach for Generalized Atocontor Testing. Implications for VIX forecast densities João Henriqe G. Mazze Department of Statistics, Universidad Carlos III de Madrid Gloria González-Rivera Department of Economics, University of California, Riverside Esther Riz Department of Statistics, Universidad Carlos III de Madrid Helena Veiga Department of Statistics, Universidad Carlos III de Madrid BRU-IUL, Institto Universitário de Lisboa Jly 27, 27 Abstract We propose an extension of the Generalized Atocontor (G-ACR) tests for dynamic specification of in-sample conditional densities and for evalation of ot-of-sample forecast densities. The new tests are based on probability integral transforms (PITs) compted from bootstrap conditional densities that incorporate parameter ncertainty. Then, the parametric specification of the conditional moments Financial spport from the Spanish Ministry of Edcation and Science, research project ECO C2-2-R (MINECO/FEDER) is acknowledged by the for athors. The forth athor also acknowledges research project ECO and FCT grant UID/GES/35/23. Gloria González-Rivera wishes to thank the Department of Statistics at UC3M for their hospitality and the financial spport of the 25 Chair of Excellence UC3M/Banco de Santander, and the UCR Academic Senate grant. We are gratefl to the participants at the New Developments in Econometrics and Time Series Workshop, Madrid, October 26, and at the IMF/IIF Workshop on Forecasting Isses on Developing Economies, Washington DC, April 27, and to seminar participants at the Management School of the University of Liverpool, for their very sefl comments. We are also thankfl to J. Mencía and E. Sentana for their help with the codes to estimate the MEM model.

2 can be tested withot relying on any parametric error distribtion yet exploiting distribtional properties of the variable of interest. We show that the finite sample distribtion of the bootstrapped G-ACR (BG-ACR) tests are well approximated sing standard asymptotic distribtions. Frthermore, the proposed tests are easy to implement and are accompanied by graphical tools that provide information abot the potential sorces of misspecification. We apply the BG-ACR tests to the Heterogeneos Atoregressive (HAR) model and the Mltiplicative Error Model (MEM) of the U.S. volatility index VIX. We find strong evidence against the parametric assmptions of the conditional densities, i.e. normality in the HAR model and semi non-parametric Gamma (GSNP) in the MEM. In both cases, the tre conditional density seems to be more skewed to the right and more peaked than either normal or GSNP densities, with location, variance and skewness changing over time. The preferred specification is the heteroscedastic HAR model with bootstrap conditional densities of the log-vix. Spplementary materials for this article are available online. Keywords: Distribtion Uncertainty; Model Evalation; Parameter Uncertainty; PIT; HAR model; Mltiplicative Error Model 2

3 Introdction Density forecasting is a very active and important area of research in the analysis of economic and financial time series. The need to consider the fll predictive density has long been recognized in the related literatre; see Tay and Wallis (2) for a srvey. A problem often faced by forecasters is testing the correct specification of a conditional forecast density; see, for example, Mitchell and Wallis(2). Appropriate tests shold take into accont that the forecast conditional distribtion is often nknown, the specification of conditional moments is also nknown, and the parameters in the conditional moments have to be estimated. Frthermore, a sefl test wold indicate the sorce of rejection of a given forecasting model, that is, whether it is rejected becase of the specification of the fnctional form of the conditional distribtion or becase of the specification of the conditional moments. Many tests for conditional forecast densities available in the literatre are based on testing a joint hypothesis of niformity and independence (i.i.d. U(,)) of the probability integral transforms (PITs) (Rosenblatt, 952) which are applicable regardless of the particlar ser s loss fnction. Diebold et al. (998) and Diebold et al. (999) introdce these tests in the econometric literatre. Intitively, the i.i.d. assmption of the PITs is related with the correct specification of the conditional moments, while the U(,) property characterizes the correct specification of the error distribtion. The PITs contain rich information on model misspecification that can be revealed by sing their histograms and atocorrelograms as sggested by Diebold et al. (998). However, it is nontrivial to develop a formal test for the joint hypothesis of independence and niformity of the PITs. The well-known Kolmogorov-Smirnov test checks niformity nder the independence assmption rather than testing both properties jointly. Conseqently, it wold easily miss the non-independent alternatives when PITs have a marginal niform distribtion. Moreover, the Kolmogorov-Smirnov test does not take into accont the impact of parameter estimation ncertainty on the asymptotic distribtion of the statistic. To solve this problem, Bai (23) proposes a Kolmogorov-Smirnov-type test based on a martingale transformation of the PITs whose asymptotic distribtion is free from the impact of parameter estimation. Yet, the test proposed by Bai (23) only checks niformity and, 3

4 conseqently, it has no asymptotic nit power if the transformed PITs are niform bt not independent; see Corradi and Swanson (26) and Rossi and Sekhposyan (23). Alternatively, Hong and Li (25) propose a nonparametric-kernel-based test with power against violations of both independence and density fnctional form, bt it depends on the choice of a bandwidth, which cold be problematic to choose in an empirical context. Instead of testing for independence and niformity of PITs, González-Rivera et al. (2) and González-Rivera and Yoldas(22) propose atocontor(acr) tests to evalate the adeqacy of the conditional density model based on the generalized errors of the model. They propose the atocontor device as a graphical tool that can be very helpfl for giding the modelling. Moreover, it permits to focs on different areas of the conditional density in order to assess those regions of interest. The ACR tests, which can be applied to both original series and model residals, have several advantages: i) they have standard convergence rates and standard limiting distribtions that deliver sperior power; ii) they are comptationally easy to implement as they are based on a conting process; iii) they do not reqire either a transformation of the original data or an assessment of the Kolmogorov goodness of fit; and iv) they explicitly accont for parameter ncertainty. Yet, they assme a parametric time-invariant fnctional form of the conditional density and, once we depart from standard densities, e.g. normal, Stdent-t, etc., the analytical expressions of the atocontors may be mathematically cmbersome to derive. This problem becomes more severe when we deal with conditional mltivariate densities. To overcome these problems, González-Rivera and Sn (25) propose the generalized atocontor (G-ACR) tests that are based on PITs instead of original observations or residals. In this way, the G-ACR tests inherit both the advantages of sing PITs and those of sing atocontors. However, the tests are still based on assming a particlar specification of the conditional density in order to compte the PITs. Therefore, when a given predictive density model is rejected, it is difficlt to disentangle whether the rejection can be attribted to the assmed fnctional form of the error distribtion (often normality) or to the specification of the conditional moments. However, it is important to nderstand whether the rejection of a forecast density is de to the often assmed normal distribtion or to the specification of the conditional moments. Frthermore, there are applications in which the conditional density does not 4

5 have a known closed-form expression. For instance, when the errors are non-gassian or when the model is non-linear, it is difficlt to obtain the fnctional form of the mlti-step predictive densities. In this paper, we propose an extension of the G-ACR tests for (in-sample) dynamic specification of a density model and for (ot-of-sample) evalation of forecast densities. Or contribtion lies on compting the PITs from a bootstrapped conditional density so that no assmption on the fnctional form of the forecast error density is needed. The only restrictions reqired on the error density are those needed to garantee that the estimator of the parameters of the conditional moments is consistent and asymptotically normal distribted. The bootstrap procedre allows for the incorporation of parameter ncertainty and can be extended to mltivariate systems. We show that the finite sample distribtions of the bootstrapped G-ACR (BG-ACR) tests are well approximated sing standard asymptotic distribtions. The proposed approach is very easy to implement and particlarly sefl to evalate forecast densities when the error distribtion is nknown. Frthermore, sing graphical devices, the procedre allows the identification of the sorce of misspecification, namely, whether it is the error distribtion or linear/non-linear dynamics. Or second contribtion is the implementation of the proposed BG-ACR tests to evalate the specification of the Heterogeneos Atoregressive (HAR) model and of the Mltiplicative Error Model (MEM), proposed to represent the dynamic evoltion of the daily forward-looking market volatility index (VIX) from the Chicago Board Options Exchange (CBOE). The VIX is important becase it is a barometer of the overall market sentiment; see Whaley (2, 29) and Diebold and Yilmaz (25) who define it as a fear index. Frthermore, it reflects both the stock market ncertainty and the expected premim from selling stock market variance in a swap contract. Finally, there is an active market on VIX derivatives. The nmber of VIX ftres contracts traded increased dramatically from abot million in 27 to abot 24 million in 22 with the largest growth occrring after 29, likely cased by the recent financial crisis; see, for example, Park (26), Song and Xi (26) and Martin (27) for some recent references on pricing VIX derivatives and Mencía and Sentana (26) for dynamic portfolio allocation for Exchange Traded Notes (ETNs) tracking short and mid-term VIX ftres indices. 5

6 The recent development of volatility-based derivative prodcts generates an interest on predictive densities of volatility. After implementing the BG-ACR tests, we show that the HAR and MEM specifications are both rejected if they do not incorporate conditional heterocedasticity. Frthermore, we also show that normality of the errors of the HAR model and the semiparametric Gamma distribtion of the errors of the MEM model are also rejected. Therestofthepaperisorganizedasfollows. Insection2, webrieflydescribe theg-acr test to make the exposition self-contained. In section 3, we provide the main contribtion with the description of the new proposed BG-ACR tests. In section 4, we analyze their in-sample finite sample performance and, in section 5, their ot-of-sample performance. In section 6, we offer an empirical application to illstrate the advantages of the BG-ACR tests; we test for the adeqacy of the HAR and MEM models to obtain forecast densities of the VIX index. Finally, we conclde in section 7. 2 The Generalized-AtoContoR (G-ACR) test We briefly describe the G-ACR test proposed by González-Rivera and Sn (25) to facilitate the reading of the forthcoming sections. Let {y t } T t= denote the random process of interest with conditional density fnction f t (y t Y t ), where Y t = (y,...,y t ) is the information set available at time t-. The random process y t may enjoy very general statistical properties, e.g. heterogeneity, dependence, etc. A conditional density model is constrcted by specifying the conditional mean, conditional variance or other conditional moments of interest, and making distribtional assmptions on the fnctional form of f t (y t Y t ). Based on the conditional model, the researcher might constrct a density forecast denoted by g t (y t Y t ) and obtain a seqence of PITs of {y t } T t= w.r.t. g t (y t Y t ) as follows t = yt g t (v t Y t )dv t. () If g t (y t Y t ) coincides with the tre conditional density, f t (y t Y t ), then the seqence of PITs, { t } T t=, mst be i.i.d. U(,); see Rosenblatt (952) and Diebold et al. (998). 6

7 Therefore, the nll hypothesis H : g t (y t Y t ) = f t (y t Y t ) is eqivalent to the nll hypothesis H : { t} T t= is i.i.d. U(,). (2) Note that, if the forecast density coincides with the tre data generating process (DGP), then it is preferred by all forecasters regardless of their particlar loss fnction; see Diebold et al. (998) and Granger and Pesaran (2a,b). In order to compte the PITs in eqation (), one needs to assme a particlar distribtion fnction for g t (y t Y t ). Simple tests of independence and niformity, sch as, the Kolmogorov-Smirnov test sffer from the problems described in the introdction. Alternatively, González-Rivera and Sn (25) propose an extension of the atocontor concepts in González-Rivera et al. (2) to evalate the properties of the PITs nder the nll hypothesis (2). Define G-ACR k,αi as the set of points in the plane ( t, t k ) sch that the sqare with αi -side and origin at (,) contains α i % of observations, i.e. G-ACR k,αi = {B( t, t k ) R 2 t α i and t k α i,s.t. : t t k α i }, (3) and the indicator series I k,α i t, which takes vale one if ( t, t k ) G-ACR k,αi and zero otherwise. If g t (y t Y t ) is a consistent estimator of f t (y t Y t ), then I k,α i t is asymptotically a Bernolli MA process whose order depends on k. The sample proportion of PIT pairs ( t, t k ) within the G-ACR k,αi cbe is given by α k,i = T t=k+ I k,α i t T k. (4) Consider the following statistic t k,αi = T k( αk,i α i ) σ αi, (5) where σ 2 α i = α i ( α i )+2α 3/2 i ( α /2 i ). González-Rivera and Sn (25) show that, nder It is possible to define other regions in the nit cbe depending on the interest of the researcher. 7

8 the nll hypothesis in (2), t k,αi is asymptotically standard normal distribted. The t-statistic in (5) is constrcted for a single fixed atocontor, α i, and a single fixed lag, k. However, it can be generalized to a set of lags with a fixed atocontor or to several atocontors with a fixed lag. In the first case, for a fixed atocontor α i, define L αi = (l,αi,...,l K,αi ) whichisak stackedvectorwithelementl k,αi = T k( α k,i α i ). Under H in (2), L α i Λ α i L αi is asymptotically χ 2 K the asymptotic covariance matrix, Λ αi, is given by: α i ( α i )+2α 3/2 i ( α /2 i ), j = k, λ j,k = 4α 3/2 i ( α /2 i ), j k. distribted, where a typical element of Alternatively, for a fixed lag k, define the vector C k = (c k,,...,c k,c ) with c k,i = T k( αk,i α i ). Once more, nder H in (2), C k Ω k C k has asymptotically a χ 2 C distribtion, where a typical element of the asymptotic covariance matrix, Ω k, is given by: α i ( α i )+2α 3/2 i ( α /2 i ), i = j, ω i,j = α i ( α j )+2α i α /2 j ( α /2 j ), i < j, α j ( α i )+2α j α /2 i ( α /2 i ), i > j. If the researcher is interested in partial aspects of the densities, sch as, a particlar collection of qantiles, it is more informative to examine the L αi statistic, which incorporates information for all desired k lags. On the other hand, if he is interested in the whole distribtion, C k collects information on all desired C atocontors for a given fixed lag k. The tests described above are based on a given known predictive density g t (y t Y t ). However, in practice, the parameters associated with the moments of this density need to be estimated. González-Rivera and Sn (25) analyze the effects of parameter estimation on the asymptotic distribtion of t k,αi, and conseqently on L αi and C k, and conclde that the corresponding adjstments to the asymptotic variance are model dependent and ths, difficlt to calclate analytically. To overcome this drawback, they propose a flly parametric bootstrap procedre to approximate the asymptotic variance based on 8

9 obtaining random extractions from the known error predictive density assmed nder the nll hypothesis. The G-ACR tests can be implemented both in-sample and ot-of-sample. González-Rivera and Sn (25) show that, when testing the ot-of-sample specification, the importance of parameter ncertainty will depend on both the forecasting scheme and the size of the estimation sample (T) relative to the forecast sample (H). When implementing the tests to check the correct specification of the ot-of-sample forecast densities, parameter ncertainty will distort the test size as long as the proportion of the ot-of-sample and in-sample sizes, H and T, respectively, is large. However, nder the assmption of T-consistent estimators, if T, H and H/T, parameter ncertainty is asymptotically negligible and no adjstment to the test is needed. Finally, note that, if any of the G-ACR tests described above rejects the nll hypothesis, there is not any indication abot whether the rejection can be attribted to an inadeqate assmption abot the error distribtion or to misspecification of the conditional moments. González-Rivera and Sn (25) point ot that the G-ACR tests are more powerfl for detecting departres from the distribtional assmption than for detecting misspecified dynamics. 3 In-sample Bootstrap G-ACR (BG-ACR) tests We propose a generalization of the G-ACR tests that allows testing for the specification of the conditional moments withot making any particlar assmption on the conditional distribtion. We also jstify heristically the asymptotic distribtion of the corresponding statistics and carry ot Monte Carlo experiments to establish the finite sample performance of the new proposed tests. Consider the following parametric location-scale model for the series of interest, y t, t =,...,T, y t = µ t +σ t ε t, (6) where µ t and σt 2 are the conditional mean and variance of y t, which are specified as parametric fnctions of the information set Y t, and ε t is a strict white noise process 9

10 with distribtion F ε, sch that E(ε t ) = and E(ε 2 t ) =. The parameters governing µ t, σt 2 and F ε garantee stationarity and satisfy the conditions reqired for their estimators to be consistent and asymptotically normal. Asymptotic normality of the parameter estimator is a reqirement for the bootstrap to be asymptotically valid for the estimation of its sample distribtion; see, for example, Hall and Yao (23). We consider a particlar specification of (6) to illstrate the proposed tests, namely the following poplar AR()-GARCH(,) model, y t = φ +φ y t +a t, (7) a t = ε t σ t, σ 2 t = ω +ω a 2 t +ω 2σ 2 t, where φ <, ω +ω 2 <, ω > and ω,ω 2 to garantee the stationarity of y t and the positiveness of the conditional variance. We consider the Qasi-Maximm Likelihood (QML) estimators of the parameters of the AR()-GARCH(,) model in (7) obtained by maximizing the Gassian log-likelihood fnction. Francq and Zakoïan (24) prove the strong consistency and asymptotic normality of the QML estimator of the ARMA-GARCH model nder finite forth order moments of the observed series. Next, we describe the proposed bootstrap algorithm to obtain in-sample one-step-ahead bootstrapconditional densities of y t in the context of the AR()-GARCH(,) model in (7). The algorithm is based on the residal bootstrap algorithms of Pascal et al. (24, 26) for the constrction of forecast densities in linear ARMA models and GARCH models, respectively. In-sample bootstrap algorithm Step Obtain the residals Estimatetheparametersofmodel(7)byatwo-stepQMLestimator: φ, φ, ω, ω and ω 2. Obtain thestandardized residals ε t = ât σ t, t = 2,...,T, where â t = y t φ φ y t, σ 2 2 = ω /( ω ω 2 ) and σ t 2 = ω + ω â 2 t + ω 2 σ t 2, for t = 3,...,T. Denote by F ε the empirical distribtion of the centered and scaled residals.

11 Step 2 Obtain bootstrap replicates of parameter estimates Fort = 3,...,T, obtainrecrsively abootstrapreplicateofy t thatmimics thedynamic dependence of the original series as follows σ 2(b) t a (b) t y (b) t = ω + ω a 2(b) t + ω 2 σ 2(b) t, (8) = ε (b) t σ (b) t, = φ + φ y (b) t +a (b) t, (9) where a (b) 2 = â 2, σ 2(b) 2 = σ 2, 2 y (b) 2 = y 2 and ε (b) t are random extractions with { } T replacement from F ε. Estimate the parameters by QML sing, obtaining φ (b), φ (b), ω (b), ω (b) and ω (b) 2. Step 3 Obtain in-sample bootstrap one-step-ahead predictive densities For t = 3,..., T, obtain in-sample one-step-ahead estimates of volatilities and observations as follows: y (b) t t=3 σ 2(b) t y (b) t = ω (b) + ω (b) (b) (b) (y t φ φ y t 2 ) 2 + ω (b) 2 σ 2(b) t, () = (b) (b) φ + φ y t +σ (b) t ε (b) t, () where σ 2(b) 2 = ω (b) /( ω (b) ω (b) 2 ) and ε (b) t replacement from F ε. are random extractions with Step 4 Repeat steps 2 and 3 for b =,...,B (). Note that, in step 2, we obtain replicates of y t that are not conditional on {y,...,y t }. In(8),σt 2 dependsona 2 t andin(9),y t dependsony t. Therefore, independent replicates of the process are generated to estimate the parameters and to obtain an estimate of their sample distribtion. However, in step 3, the bootstrap replicates, σ 2 t and y t, in () and () respectively, are obtained incorporating the parameter ncertainty throgh the bootstrap estimates of the parameters bt always conditional on the original data {y,...,y t }. In this way, at each moment of time, t = 3,...,T, the above algorithm

12 generates B () bootstrap replicates of y t conditional on Y t incorporating parameter ncertainty and avoiding any specific assmption abot the distribtion of ε t. In order to decide the nmber of bootstrap replicates that garantees an appropriate estimate of the predictive density, one can implement the procedre proposed by Andrews and Bchinsky (2). Note that a non-linear GARCH model will reqire a larger nmber of bootstrap replicates than a linear model. In-sample PITs can be easily compted as follows t = B () (y (b) B () t < y t ), (2) b= where (.) is the indicator fnction which takes vale when the argment is tre and zero otherwise. After compting the corresponding indicators, I k,α i t, the sample proportions, α k,i, can be calclated as in (4). Finally, the t k,αi, L αi and C k statistics can be calclated as explained in the previos section. It is worth noting that the proposed procedre to obtain in-sample bootstrap conditional densities, and the conseqent BG-ACR statistics to evalate them, can be applied to any other parametric specifications of the conditional mean and conditional variance (and any other higher moments) as far as a consistent and asymptotically normal estimator of the parameters is available; see, for example, Mika and Saikkonen (2) who prove the strong consistency and asymptotic normality of the Gassian QML estimator allowing both the conditional mean and the conditional variance to be nonlinear. In order to illstrate how the proposed procedre works, we have generated a time series of size T = 5 from the following homoscedastic AR() model: y t = φ y t +ε t, (3) with φ = (.5,.95) and i.i.d. ε t either N(,), or centered and standardized Stdent-5, or χ 2 (5). In each case, an AR() model is fitted to the artificial series with the parameters estimated by QML. Then, the in-sample PITs are compted (i) assming normal errors as in González-Rivera and Sn(25) and(ii) implementing the bootstrap algorithm described above based on B () = 999 replicates; see Pascal et al. (24, 26) and Pan and Politis 2

13 (26) for the same nmber of replicates. In Figre, we plot the atocontors for α i =.2 and.8 together with the pairs ( t, t ) for the AR() model with φ =.5 and ε t N(,) (first row); φ =.5 and ε t Stdent-5 (second row); φ =.5 and ε t χ 2 (5) (third row); and φ =.95 and ε t χ 2 (5) (forth row). Note that, when the PITs are compted sing the bootstrap densities (first colmn), they are niformly distribted on the srface regardless of the tre error distribtion of the nderlying DGP. Therefore, they sggest that the fitted AR() model is adeqate. However, when the PITs are compted as in the G-ACR procedre (second colmn), assming normality, they are not niformly distribted nless the errors are Gassian. In this case, when the model is rejected, there is not indication abot whether the rejection is coming from the misspecification of the conditional mean or from a misspecified fnctional form of the error distribtion. 3

14 t t t t t.6 t t t t t t t t t t t Figre : Pairs (t, t ) and atocontors for the estimated AR() model with T = 5. ACR2%, corresponds to the black box and the ACR8%, to the red box. The DGPs are the AR() model with: φ =.5 and εt N (, ) (first row); φ =.5 and εt Stdent-5 (second row); φ =.5 and εt χ2(5) (third row); and φ =.95 and εt χ2(5) (forth row). The PITs were compted sing the bootstrap algorithm with B () = (first colmn), or assming Gassian errors (second colmn). 4

15 Consider now the following three DGPs, from which we generate three time series of size T = 5 y t =.3y t +.6y t 2 +ε t, (4).5y t +ε t, for t < T/2, y t = +.5y t +ε t, for t T/2, (5) y t =.5y t +ε t σ t, (6) σ 2 t =.5+.5ε 2 t σ 2 t +.45σ 2 t, with ε t defined as above. We fit an AR() model to each of the simlated series and estimate its parameters by QML. As above, we compte the PITs both assming normal errors and sing the proposed bootstrap procedre. In Figre 2, we plot the atocontors for α i =.2 and.8 together with the pairs ( t, t ) when the DGP is the AR(2) model in (4) with χ 2 (5) errors (first row); the AR() model with strctral break in the mean in (5) with ε t χ 2 (5) (second row); the GARCH model in (6) with normal errors (third row); and the GARCH model in (6) with χ 2 (5) errors (forth row). We observe that, when the PITs are based on bootstrap densities (first colmn), they sggest the sorce of the misspecification. In the first row, when the AR() model is fitted to the AR(2) series, we observe a linear relation between the PITs, which tend to grop arond one of the diagonals of the nit-sqare. In the second row, when the DGP is the AR() model with a break in the mean, the PITs do not show any particlar linear or non-linear relationship bt they are concentrated on the top-right corner of the nit-sqare. Finally, when the DGP is the AR()-GARCH(,) model, we observe a non-linear relation between the PITs, which are more concentrated towards the for corners of the nit-sqare. Frthermore, in this last case, the atocontor plots are very similar regardless of the error distribtion of the DGP. Comparing the bootstrap-based PITs with those obtained sing G-ACR assming a normal density (second colmn), the rejection of the fitted AR() model is also evident. However, there is not an obvios indication of the sorce of the misspecification. 5

16 t t t t t.6 t t t t t t t t t t.2.4 t Figre 2: Pairs (t, t ) and atocontors for estimated AR() model with T = 5. ACR2%, corresponds to the black box and the ACR8%, to the red box. The DGPs are: AR(2) with εt χ2(5) (first row); AR() model with break in the mean with εt χ2(5) (second row); AR()-GARCH(,) model with εt N(,) (third row); and AR()-GARCH(,) model with εt χ2(5) (forth row). The PITs were compted sing the bootstrap algorithm with B () = (first colmn), or assming Gassian errors (second colmn). 6

17 Theasymptoticdistribtionsofthet k,αi, L αi andc k statisticsdependontheasymptotic validity of the residal bootstrap algorithm described above which has been established by Pascal et al. (24) in the context of obtaining predictive densities of linear ARMA models. However, as far as we know, there is not a formal proof of the validity of the procedre to constrct predictive densities of nonlinear GARCH models. In order to show that the algorithm is asymptotically valid, one needs first to show that the bootstrap procedre in step 2, generates asymptotically valid estimates of the model parameters. When implemented in GARCH models, Hidalgo and Zaffaroni (27) show the first order validity of the bootstrap QML estimator of the parameters of an ARCH( ) process characterized by a particlar decay in the ARCH parameters. 2 If the bootstrap procedre is asymptotically valid for the estimation of the parameters, sing the argments in Pascal et al. (24) and Reeves (25), one can establish its validity for the predictive densities and, conseqently, the distribtion of α k,i shold be as in (5) with the asymptotic variance corrected to take into accont parameter ncertainty. 3 Following the sggestion of González-Rivera and Sn (25), the variance of α k,i is approximated sing a bootstrap procedre to take into accont parameter ncertainty. B (2) bootstrap replicates, {y (b) t } T t= are generated as in (9) and α (b) k,i is obtained sing the bootstrap series as if they were the original series. The bootstrap variance of α k,i is given by σ 2 α i = B (2) B (2) b= and the corresponding corrected t-statistic is α (b) k,i B (2) B (2) b= α (b) k,i 2, (7) t k,α i = ( α k,i α i ) σ α i, (8) 2 Shimiz (2, 23, 24) prove the consistency of the bootstrap QML estimator in the context of an AR()-ARCH() model. However, the residal bootstrap considered by Shimiz (2, 23, 24) is not exactly the same as that considered in this paper. All the trajectories share the same estimated conditional mean and variance when generating bootstrap replicates to estimate the parameters. It is important to point ot that Corradi and Iglesias (28) cast some dobts on the asymptotic validity of the residal bootstrap described in step 2. Alternatively, they show that a block bootstrap based on resampling the likelihood as proposed by Gonçalves and White (24) is asymptotically valid. Therefore, in step 2 of the algorithm described above, one can consider sing this block bootstrap instead of the residal bootstrap. 3 Monte Carlo reslts on the size distortions of the t-statistic when the asymptotic variance is compted as in (5) are available pon reqest. 7

18 whichasymptoticallyhasan(,)distribtion. Inthispaper, resltsarebasedonb (2) =5 bootstrap replicates to compte σ α i ; see González-Rivera and Sn (25). Note that the nmber of replicates needed to estimate standard errors is smaller than that reqired to estimate intervals; see Efron (987). Obviosly, the variances and covariances of the portmantea statistics can also be compted sing the same argments. In particlar, a typical element of the covariance matrix of L αi, say λ j,k, is obtained as follows: λ j,k = σ 2 α i, if j = k, ( )( ) α (b) j,i B (2) α (b) B (2) j,i α (b) k,i B (2) α (b) B (2) k,i, if j k. b= b= B (2) B (2) b= (9) Similarly, a typical element of the covariance matrix of C k, say ωi,j, is obtained analogosly to (9) as follows: ωi,j = σ 2 α i, if i = j, ( )( ) B (2) α (b), if i j. B (2) b= α (b) k,i B(2) B (2) b= α (b) k,i k,j B(2) B (2) b= α (b) k,j (2) 4 Finite sample performance of in-sample tests We perform Monte Carlo simlations to assess the finite sample properties of the proposed statistics. For the size assessment, the DPG is a linear AR(). We consider a model far from the non-stationary region and another one near the non-stationary region with different error distribtions. For the power assessment, we consider linear and non-linear alternatives. The nmber of Monte Carlo replicates is R = and the sample size T = 5,, 3, and 5. The nmber of bootstrap replicates is B () =, except for T = 5, when we se B () = 2. Finally, the nmber of bootstrap replicates sed to compte the variance of α k,i, L αi and C k is B (2) = 5. 8

19 4. Stdying the size To investigate the size properties of the tests, we consider as DGP the AR() in (3). For each Monte Carlo replicate, we compte the proportions α k,i, for k =,...,5, and their bootstrap variances. Then, we compte the Monte Carlo averages and standard deviations of α k,i, together with the averages of the bootstrap standard deviations and the percentage of rejections of the nll hypothesis when the nominal size of the test is 5%. Table reports the Monte Carlo reslts for k = when φ =.95 and the errors are χ 2 (5). We observe that, even for the smallest sample size of T = 5, the Monte Carlo averages of α k,i are rather close to α i and that, for moderate sample sizes, the average of the bootstrap standard deviations is a good approximation to the Monte Carlo standard deviation of α k,i. For relatively small sample sizes, the bootstrap standard deviations tend to overestimate the empirical standard deviations of α k,i, mainly for the largest qantiles. Conseqently, the size of the t,αi statistic is smaller than the nominal. As the sample size increases, the percentage of rejections becomes rather close to the 5% nominal level. We also analyze the finite sample performance of the two portmantea tests. Table 2 reports the Monte Carlo percentage of rejections of L 5 α i (adding p the information over the first five lags) and of C (adding information of the thirteen qantiles considered) for the same model considered in Table. Regarding L 5 α i, we observe that, the Monte Carlo percentage of rejections is very close to the nominal size with a tendency to over-reject for the largest qantiles. Regarding C, we observe that it nder-rejects when the sample size is not large enogh, and over-rejects for very large samples. Smmarizing, asymptotic normality is a good approximation to the finite sample performance of the proposed BG-ACR test nder the nll of correct specification as far as we do not consider extreme atocontors. This conclsion is valid regardless of the particlar error distribtion and the persistence properties of the conditional mean. 4 4 Reslts for the AR() model with φ =.5 and Gassian errors are reported in Tables A and B of the spplementary material. The conclsions are the same. 9

20 Table 2 Monte Carlo size reslts for t,αi. The DGP is y t =.95y t +ε t, with ε t χ 2 (5) and the nominal size is 5%. T α i α k,i std (.22) (.48) (.67) (.89) (.98) (.2) (.97) (.94) (.8) (.64) (.43) (.34) (.23) size α k,i std (.4) (.32) (.45) (.6) (.64) (.67) (.62) (.57) (.49) (.38) (.27) (.2) (.2) size α k,i std (.7) (.7) (.24) (.3) (.33) (.32) (.32) (.28) (.24) (.8) (.3) (.9) (.6) size α k,i std (.4) (.9) (.2) (.6) (.7) (.7) (.6) (.5) (.2) (.9) (.6) (.4) (.3) size α k,i std (.2) (.4) (.5) (.7) (.8) (.7) (.7) (.6) (.5) (.4) (.2) (.2) (.) size

21 Table 2 2 Monte Carlo size reslts for L 5 α i and C statistics. The DGP is y t =.95y t +ε t, ε t χ 2 (5). The nominal size is 5%. L 5. L 5.5 L 5. L 5.2 L 5.3 L 5.4 L 5.5 L 5.6 L 5.7 L 5.8 L 5.9 L 5.95 L 5.99 C

22 4.. Stdying the power To stdy the finite sample power of the tests, we generate replicates sing the models in eqations (5) and (6). In both cases, we fit an AR() model. Under the nll hypothesis, we test the correct specification of the AR() model withot drift. For the DGP in (5), we analyze their power against breaks in the conditional mean while for the DGP in (6), we stdy their power against misspecification in the conditional variance. In Tables 3 and 4, we report the power reslts of the test t,αi for each of these two DGPs, respectively, and in Table 5, we report the power reslts corresponding to the portmantea tests. When the DGP has a break in the conditional mean (Table 3), the tests have high power (7-%) for sample sizes of 3 and above. When the DGP is an AR()-GARCH(,) model (Table 4), we observe that the power of t,αi is the highest in the extreme % and 5% atocontors. This is consistent with the reslts that we have explained in Figre 2, where the pairs of PITs tend to concentrate into the corners of the nit-sqare. These reslts sggest that we need larger sample sizes for the tests to have power against misspecification in the conditional variance. 5 Regarding the portmantea tests (Table 5), both L 5 α i and C are very powerfl for detecting breaks in the conditional mean when the sample size is 3 and above. Detecting misspecification in the conditional variance is more difficlt in small samples and we need sample sizes beyond observations to obtain high power. As with the t,αi, the power of L 5 α i is higher in the extreme atocontors. 6 5 Note that this reslt cold be expected as inference in nonlinear GARCH models reqire largesamples. 6 Reslts on the power when the DGP is the AR(2) model in (4) are reported in Tables C and D of the spplementary material. The proposed tests are very powerfl even for small sample sizes. 22

23 Table 3 23 Monte Carlo power reslts for t,αi. The DGP is the AR() model with break in the mean and ε t N(,). The nominal size is 5%. T α i α k,i std (.5) (.6) (.26) (.4) (.49) (.55) (.59) (.59) (.6) (.52) (.45) (.38) (.27) power α k,i std (.4) (.3) (.9) (.27) (.33) (.39) (.42) (.4) (.39) (.36) (.29) (.26) (.7) power α k,i std (.3) (.7) (.) (.6) (.2) (.22) (.24) (.25) (.24) (.2) (.8) (.4) (.8) power α k,i std (.2) (.4) (.6) (.9) (.) (.2) (.3) (.3) (.3) (.2) (.9) (.8) (.4) power α k,i std (.) (.2) (.3) (.4) (.5) (.5) (.6) (.6) (.6) (.5) (.4) (.3) (.2) power

24 Table 4 24 Monte Carlo power reslts for t,αi. The DGP is the AR()-GARCH(,) model in (6) and ε t N(,). The nominal size is 5%. T α i α k,i std (.9) (.39) (.56) (.78) (.89) (.96) (.93) (.86) (.77) (.6) (.4) (.3) (.9) power α k,i std (.4) (.29) (.43) (.6) (.7) (.75) (.72) (.64) (.5) (.38) (.26) (.8) (.) power α k,i std (.9) (.22) (.3) (.4) (.46) (.48) (.45) (.39) (.3) (.2) (.2) (.8) (.5) power α k,i std (.6) (.) (.6) (.23) (.26) (.27) (.26) (.2) (.7) (.2) (.6) (.4) (.2) power α k,i std (.3) (.7) (.) (.4) (.6) (.7) (.6) (.5) (.2) (.7) (.3) (.2) (.) power

25 Table 5 25 Monte Carlo power reslts for L 5 α i and C statistics. The DGPs are: AR() model with break in the mean (Panel A) and AR()-GARCH(,) (Panel B). The nominal size is 5%. L 5. L 5.5 L 5. L 5.2 L 5.3 L 5.4 L 5.5 L 5.6 L 5.7 L 5.8 L 5.9 L 5.95 L 5.99 C 3 Panel A Panel B

26 5 Ot-of-sample h-step-ahead Bootstrap G-ACR (BG-ACR) tests We extend the procedres and tests described in the previos section to obtain ot-of-sample h-step-ahead densities. The in-sample bootstrap algorithm can also be applied to obtain bootstrap replicates of the ot-of-sample mlti-step-ahead observations. Then, the corresponding PITs and indicators can be compted. In order to compte the proportion α k,i, it is necessary to obtain (H h + ) h-step-ahead bootstrap forecast densities. If the parameters are not re-estimated each time a new observation is available, then the in-sample algorithm can be implemented as described in Section 3 with step 3 modified as follows: Step 3. Obtain ot-of-sample h-step-ahead bootstrap forecast densities Forh =,2,...andj =,...,H h obtainot-of-sampleh-step-aheadconditional estimates of volatilities and observations as follows: σ 2(b) T+h+j T+j y (b) T+h+j T+j = ω (b) + ω (b) (y (b) T+h +j T+j µ (b) φ (b) y T+h 2+j T+j ) 2 + ω (b) 2 σ 2(b) T+h +j T+j, = µ (b) + φ (b) y (b) T+h +j T+j +σ (b) T+h+j T+j ε (b) T+h, (2) where y (b) T+i T = y T+i when i and σ 2(b) i i = + ω (b) i 3 ω (b) ω (b) ω (b) 2 i = T,T +,...,T +H. j= ω (b)j 2 [ (y i j µ (b) φ (b) y i j 2 ) 2 ] ω (b) ω (b) ω (b) 2 for follows At each moment T + j, j = h,...,h, we compte ot-of-sample mlti-period PITs as T+j T+j h = B () (y (b) B () T+j T+j h < y T+j). b= Note that the PITs based on h-step-ahead density forecasts will generally follow a moving average process of order h. When h >, nder the nll that the predictive density coincides with the tre density, the distribtional featres of the PITs are not well defined. As a reslt, it is only possible to test the nll of a well behaved density forecast jointly with an assmed model of the process driving the associated h-step-ahead PITs. Alternatively, 26

27 one can choose PITs separated by h periods to ensre an ncorrelated sample. This procedre may significantly redce the evalation sample when h is relatively large. In this case, the procedre can be implemented in several ncorrelated sb-samples of forecasts that are h periods apart and then se Bonferroni methods to obtain a joint test withot discarding observations; see, for example, Diebold et al. (998), Manzan and Zerom (28) and Rossi and Sekhposyan (26) among others. Using the ncorrelated PITs { T+ht T+h(t ) } [H/h] t=, we compte the corresponding indicators, I k,α i T+ht, and the proportions α k,i = [H/h] t=k+ I k,α i T+ht H/h k. Finally, the t-statistic is given by t k,αi = H/h k( αk,i α i ) σ αi, where σ 2 α i is defined as in (5). Note that σ 2 α i can be estimated either as in expression (5) or by bootstrapping. As mentioned above, when testing the in-sample specification, ignoring parameter ncertainty may case severe distortions in the size of the tests. However, when testing the ot-of-sample specification, the importance of parameter ncertainty decreases as far H/T when T and H. Therefore, if H is small relative to T, one can compte the variance σ 2 α i by sing the asymptotic expression. As an illstration of the ot-of-sample one-step-ahead performance of the tests, we generate R = replicates from the AR() model in expression (3) with φ =.95 and ε t N(,). The model is estimated by OLS sing T = 5,, 3, and 5 observations and H = 5 and 5 ot-of-sample one-step-ahead densities. Their corresponding PITs are obtained sing the bootstrap procedre. The variance of α k,i and the covariances in Λ αi and Ω k are compted by bootstrapping. 7 In Table 6, we report the 7 Reslts based on the asymptotic expression of the variances and covariances are very similar when H = 5 and T = (H/T =.5) or T = 5 (H/T =.). When H = 5, the reslts are similar if T = 5 (H/T =.). As mentioned above, in these cases, the parameter ncertainty is irrelevant. These reslts are available pon reqest. 27

28 size of the corresponding L 5 α i and C 3 test statistics for H = 5 and H = 5. Increasing H improves the size properties of the tests as far as the ratio H/T is still small. For small estimation samples, the tests tend to be oversized bt the size is corrected when the estimation and evalation samples are larger. 8 8 Reslts for the t-tests are reported in Table E of the spplementary material. For small estimation sizes, the test tends to be oversized for the middle atocontors. When T is relatively large and H/T is small, the empirical size is abot 5%. 28

29 Table 6 29 Monte Carlo size reslts for ot-of-sample L 5 α i and C statistics. The DGP is y t =.95y t +ε t and ε t N(,). The nominal size is 5%, H = 5 (Panel A) and H = 5 (Panel B). L 5. L 5.5 L 5. L 5.2 L 5.3 L 5.4 L 5.5 L 5.6 L 5.7 L 5.8 L 5.9 L 5.95 L 5.99 C 3 T Panel A Panel B

30 Finally, we stdy the finite sample power of the ot-of-sample one-step-ahead tests. With this prpose, we generate R = replicates from the AR(2) model in (4) and from the AR()-GARCH(,) model in (6). Under the nll hypothesis, we estimate an AR()process withot drift. InTable 7, we report thepower of thet,αi test when thedgp is the AR(2) model and H = 5. For small estimation samples (T = 5,), the power is high in the middle 2%-7% atocontors. As the estimation sample grows, the power reaches for most atocontors. When the information is accmlated either over several lags or over several qantiles, as in the L 5 α i and C 3 tests (Table 9, Panel A), the power is very high even for small estimation samples. We report the power reslts of the t,αi tests corresponding to the AR()-GARCH(,) in Table 8 with H = 5. We observe a similar behavior as in the in-sample tests. The information on heteroscedasticity is contained in the lower % and 5% atocontors and large estimation samples are reqired. In Panel B of Table 9, we observe an acceptable power of arond 6% for the C 3 statistic for estimation samples of T = 3 and above. The test L 5 α i delivers similar power for the lowest % atocontor and for the highest 95-99% atocontors. It is important to note that in-sample tests are expected to be more powerfl than ot-of-sample tests; see, for example, Inoe and Kilian (25). 3

31 Table 7 3 Monte Carlo power reslts for ot-of-sample t,αi. The DGP is the AR(2) model and ε t N(,). The nominal size is 5% and H = 5. T α i α k,i std (.) (.3) (.8) (.2) (.38) (.59) (.79) (.95) (.2) (.98) (.83) (.68) (.49) power α k,i std (.) (.3) (.7) (.9) (.32) (.47) (.59) (.69) (.76) (.73) (.6) (.47) (.3) power α k,i std (.) (.2) (.6) (.5) (.23) (.33) (.4) (.46) (.49) (.47) (.38) (.28) (.5) power α k,i std (.) (.2) (.6) (.2) (.9) (.26) (.33) (.38) (.39) (.36) (.29) (.22) (.) power α k,i std (.) (.2) (.5) (.2) (.8) (.23) (.28) (.33) (.33) (.3) (.25) (.8) (.9) power

32 Table 8 32 Monte Carlo power reslts for ot-of-sample t,αi. The DGP is the AR()-GARCH(,) model with ε t N(,). The nominal size is 5% and H = 5. T α i α k,i std (.22) (.36) (.47) (.63) (.82) (.98) (.9) (.4) (.) (.) (.83) (.69) (.54) power α k,i std (.9) (.3) (.39) (.53) (.64) (.79) (.87) (.9) (.9) (.84) (.68) (.54) (.37) power α k,i std (.5) (.25) (.32) (.4) (.48) (.58) (.66) (.69) (.69) (.65) (.52) (.4) (.24) power α k,i std (.2) (.9) (.24) (.3) (.35) (.4) (.46) (.49) (.5) (.49) (.4) (.3) (.7) power α k,i std (.2) (.8) (.22) (.26) (.3) (.36) (.4) (.45) (.47) (.45) (.36) (.27) (.3) power

33 Table 9 33 Monte Carlo power reslts for ot-of-sample L 5 α i and C statistics with H = 5. The DGPs are: the AR(2) model with ε t N(,) (Panel A); and the AR()-GARCH(,) model with ε t N(,) (Panel B). The nominal size is 5%. L 5. L 5.5 L 5. L 5.2 L 5.3 L 5.4 L 5.5 L 5.6 L 5.7 L 5.8 L 5.9 L 5.95 L 5.99 C 3 Panel A Panel B

34 6 Empirical application: Modeling VIX There is an increasing interest in modeling and forecasting the volatility index VIX from the CBOE. The VIX was originally compted as the weighted average of the implied volatilities from eight at-the-money call and pt options of the S&P index with an average time to matrity of 3 days. In 23, the VIX was entirely revised by changing the reference index to the S&P5 index, taking into accont a wide range of strike prices with the same time to matrity, and freeing its calclation from any specific option pricing model; see Whaley (29) for a history of the VIX and Fernandes et al. (24) for a detailed description of the VIX calclation. The recent development of volatility-based derivative prodcts generates an interest on predictive densities of volatility. Intitively, risk averse investors mst take into accont not only the expected vale of the payoffs, which can be obtained from the conditional mean forecasts, bt also the risk involved, which necessarily depends on featres of the conditional density. In the context of VIX, Konstantinidi and Skiadopolos (2) implement the bootstrap procedre of Pascal et al. (24) to obtain forecast intervals for the VIX that are then sed in a trading strategy. Konstantinidi et al. (28) and Psaradellis and Sermpinis (26) also compare several specifications of the VIX for trading prposes. It is commonly accepted that the VIX display long-memory; see, for example, Konstantinidi et al. (28) and Fernandes et al. (24). Conseqently, several athors propose variants of the simple and easy-to-estimate long-memory Heterogeneos Atoregressive (HAR) model of Corsi (29) to represent and predict the VIX; see Fernandes et al. (24), Caporin et al. (26) and Psaradellis and Sermpinis (26). Alternatively, Mencía and Sentana (26) propose modeling the persistence of the VIX sing the Mltiplicative Error Model (MEM) of Engle and Gallo (26) with a semi-nonparametric expansion of the Gamma distribtion. In this section, we implement the BG-ACR tests to one-step-ahead in-sample conditional densities obtained after fitting the HAR and MEM models to V t, the daily VIX index, observed from Janary 2, 99 to Janary 5, 23 with a total of 587 observations. Fernandes et al. (24), who analyze the same series, show that the nll hypothesis of a nit-root is clearly rejected and find strong evidence of long-memory. Conseqently, the 34