Resampling in State Space Models

Transcription

1 Resampling in State Space Mdels David S. Stffer Department f Statistics University f Pittsburgh Pittsburgh, PA USA Kent D. Wall Defense Resurces Management Institute Naval Pstgraduate Schl Mnterey, CA USA Abstract. Resampling the innvatins sequence f state space mdels has prved t be a useful tl in many respects. Fr example, while under general cnditins, the Gaussian MLEs f the parameters f a state space mdel are asympttically nrmal, several researchers have fund that samples must be fairly large befre asympttic results are applicable. Mrever, prblems ccur if the any f parameters are near the bundary f the parameter space. In such situatins, the btstrap applied t the innvatin sequence can prvide an accurate assessment f the sampling distributins f the parameter estimates. We have als fund that a resampling prcedure can prvide insight int the validity f the mdel. In additin, the btstrap can be used t evaluate cnditinal frecast errrs f state space mdels. The key t this methd is the derivatin f a reverse-time innvatins frm f the state space mdel fr generating cnditinal data sets. We will prvide sme theretical insight int ur prcedures that shw why resampling wrks in these situatins, and we prvide simulatins and data examples that demnstrate ur claims. Key wrds. ARMAX mdels, Btstrap, Finite sample distributins, Frecasting, Innvatins filter, Kalman filter, Reverse-time state space mdel, Stchastic regressin, Stchastic vlatility. 1 Intrductin A very general mdel that seems t subsume a whle class f special cases f interest is the state space mdel r the dynamic linear mdel, which was intrduced in Kalman (1960) and Kalman and Bucy (1961). Althugh the mdel was riginally develped as a methd primarily fr use in aerspace-related research, it has been applied t mdeling data frm such diverse fields as ecnmics (e.g. Harrisn and Stevens, 1976, Harvey and Pierse, 1984, Harvey and Tdd, 1983, Kitagawa and Gersch 1984, Shumway and Stffer, 1982), medicine (e.g. Jnes, 1984) and mlecular bilgy (e.g. Stultz, et al, 1993). An excellent mdern treatment f time series analysis based n the state space mdel is the text by Durbin and Kpman (2001). We nte, in particular, that ARMAX mdels can be written in state space frm (see e.g. Shumway and Stffer, 2000, 4.6), s anything we say and d here regarding state space mdels applies equally t ARMAX mdels. Here, we write the state space mdel as x t+1 =Φx t +Υu t + w t t =0, 1,..., n (1) y t = A t x t +Γu t + v t t =1,..., n (2) where x t represents the p-dimensinal state vectr, and y t represents the q-dimensinal bservatin vectr. In the state equatin (1), the initial state x 0 has mean µ 0 and variance-cvariance matrix Chapter 9 f State Space and Unbserved Cmpnent Mdels: Thery and Applicatins, A. Harvey, S.J. Kpman and N. Shephard (Eds). Cambridge University Press, 2004

2 Σ 0 ;Φisp p, Υisp r, andu t is an r 1 vectr f fixed inputs. In the bservatin equatin (2), A t is q p and Γ is q r. Here, w t and v t are white nise series (bth independent f x 0 ), with var(w t )=Q, var(v t )=R, but we als allw the state nise and bservatin nise t be crrelated at time t; thatis,cv(w t,v t )=S, and zer therwise. Nte, S is a p q matrix. Thrughut, we assume the mdel cefficients and the crrelatin structure f the mdel are uniquely parameterized by a k 1 parameter vectr Θ; thus, Φ = Φ(Θ), Υ = Υ(Θ), Q = Q(Θ), A t = A t (Θ), Γ = Γ(Θ), R = R(Θ), and S = S(Θ). We dente the best linear predictr f x t+1 given the data {y 1,...,y t } as x t t+1, and dente the cvariance matrix f the predictin errr, (x t+1 x t t+1 ), as P t t+1. The Kalman filter (e.g. Andersn and Mre, 1979) can be used t btain the predictrs and their cvariance matrices successively as new bservatins becme available. The innvatin sequence, {ɛ t ; t =1,...,n}, is defined t be the sequence f errrs in the best linear predictin f y t given the data {y 1,...,y t 1 }. The innvatins are ɛ t = y t A t xt t 1 Γu t, t =1,...,n, (3) where the innvatin variance-cvariance matrix is given by Σ t = A t Pt t 1 A t + R, t =1,...,n. (4) The innvatins frm f the Kalman filter, fr t =1,...,n, is given by the fllwing equatins with initial cnditins x 0 1 =Φµ 0 +Υu 0 and P 0 1 =ΦΣ 0Φ + Q: x t t+1 = Φxt t 1 +Υu t + K t ɛ t, (5) Pt+1 t = ΦPt t 1 Φ + Q K t Σ t K t, (6) K t = (ΦPt t 1 A t + S)Σ 1 t. (7) In this article, we will wrk with the standardized innvatins e t =Σ 1/2 t ɛ t, (8) s we are guaranteed these innvatins have, at least, the same first tw mments. In (8), Σ 1/2 t dentes the unique square rt matrix f Σ t defined by Σ 1/2 t Σ 1/2 t =Σ t. We nw define the (p+q) 1 vectr [ x t t+1 ξ t = Cmbining (3) and (5) results in a vectr first-rder equatin fr ξ t given by where F t = y t ]. ξ t = F t ξ t 1 + Gu t + H t e t, (9) [ ] [ ] Φ 0 Υ, G =, H t = A t 0 Γ [ Kt Σ 1/2 ] t Σ 1/2. t Estimatin f the mdel parameters Θ is accmplished by Gaussian quasi-maximum likelihd. The innvatins frm f the Gaussian likelihd (ignring a cnstant) is ln L Y (Θ) = 1 2 = 1 2 n t=1 n ( t=1 ( ) ln Σ t (Θ) + ɛ t (Θ) Σ t (Θ) 1 ɛ t (Θ) ) ln Σ t (Θ) + e t (Θ) e t (Θ), (10) 1

3 where L Y (Θ) dentes the likelihd f Θ given the data y 1,...,y n assuming nrmality; nte that we have emphasized the dependence f the innvatins n the parameters Θ. We stress the fact that it is nt necessary fr the data t be Gaussian t cnsider (10) as the criterin functin t be used fr parameter estimatin. Furthermre, under certain rare cnditins, the Gaussian quasi-mle f Θ when the prcess is nn-gaussian is asympttically ptimal; details can be fund in Caines (1988, Chapter 8). 2 Assessing the Finite Sample Distributin f Parameter Estimates Althugh, under general cnditins (which we assume t hld in this sectin), the MLEs f the parameters f the mdel, Θ, are cnsistent and asympttically nrmal, time series data are ften f shrt r mderate length. Several researchers have fund evidence that samples must be fairly large befre asympttic results are applicable (Dent and Min, 1978; Ansley and Newbld, 1980). Mrever, it is well knwn that prblems ccur if the parameters are near the bundary f the parameter space. In this sectin, we discuss an algrithm fr btstrapping state space mdels t assess the finite sample distributin f the mdel parameters. This algrithm and its justificatin, including the nn-gaussian case, alng with examples and simulatins, can be fund in Stffer and Wall (1991). Let Θ dente the Gaussian quasi-mle f Θ, that is, Θ =argmax Θ L Y (Θ), where L Y (Θ) is given in (10); f curse, if the prcess is Gaussian, Θ isthemle.letɛ t ( Θ) and Σ t ( Θ) be the innvatin values btained by running the filter under Θ. Once this has been dne, the btstrap prcedure is accmplished by the fllwing steps. 1. Cnstruct the standardized innvatins e t ( Θ) = Σ 1/2 t ( Θ)ɛ t ( Θ). 2. Sample, with replacement, n times frm the set {e 1 ( Θ),..., e n ( Θ)} t btain {e 1,..., e n },a btstrap sample f standardized innvatins. 3. T cnstruct a btstrap data set {y 1,..., y n },slve(9)usinge t in place f e t ; that is, slve ξ t = F t ( Θ)ξ t 1 + G( Θ)u t + H t ( Θ)e t, (11) fr t =1,...,n. The exgenus variables u t and the initial cnditins f the Kalman filter remain fixed at their given values, and the parameter vectr is held fixed at Θ. Nte that a btstrapped bservatin y t is btained frm the final q rws f the (p + q) 1 vectr ξ t. Because f startup irregularities, it is smetimes a gd idea t set y t y t fr the first few values f t, sayt =1, 2,...,t 0,wheret 0 is small, and t sample frm {e t0 +1( Θ),..., e n ( Θ)}. That is, d nt btstrap the first few data pints; typically setting t 0 t4r5willsuffice. 4. Using the btstrap data set {y t ; t =1,..., n}, cnstruct a likelihd, L Y (Θ), and btain the MLE f Θ, say, Θ. 2

4 5. Repeat steps 2 thrugh 4, a large number, B, f times, btaining a btstrapped set f parameter estimates { Θ b ; b =1,..., B}. The finite sample distributin f ( Θ Θ) may be apprximated by the distributin f ( Θ b Θ), fr b =1,..., B. 2.1 Stchastic Regressin An interesting applicatin f the state-space mdel was given in Newbld and Bs (1985, pp ). Of the several alternative mdels they investigate, we fcus n the ne specified by their equatins (4.7a) and (4.7b). Their mdel has ne utput variable, the nminal interest rate recrded fr three-mnth treasury bills, y t. The utput equatin is specified by y t = α + β t z t + v t, where z t is the quarterly inflatin rate in the Cnsumer Price Index, α is a fixed cnstant, β t is a stchastic regressin cefficient, and v t is white nise with variance σ 2 v. The stchastic regressin term, which cmprises the state variable, is specified by a first-rder autregressin, (β t+1 b) =φ(β t b)+w t, where b is a cnstant, and w t is white nise with variance σw. 2 The nise prcesses, v t and w t,are assumed t be uncrrelated. Using the ntatin f the state space mdel (1) and (2), we have in the state equatin, x t = β t, Φ=φ, u t 1,Υ=(1 φ)b, Q = σw, 2 and in the bservatin equatin, A t = z t,γ=α, R = σv, 2 and S =0. TheparametervectrisΘ=(φ, α, b, σ w,σ v ). We cnsider the first estimatin exercise reprted in Table 4.3 f Newbld and Bs. This exercise cvers the perid frm the first quarter f 1953 thrugh the secnd quarter f 1965, n =50 bservatins. We repeat their analysis s ur results can be cmpared t their results. In additin, we fcus n this analysis because it demnstrates that the btstrap applied t the innvatin sequence can prvide an accurate assessment f the sampling distributins f the parameter estimates when analyzing shrt time series. Mrever, this analysis demnstrates that a resampling prcedure can prvide insight int the validity f the mdel. The results f the Newtn Raphsn estimatin prcedure are listed in Table 1. The MLEs btained in Newbld and Bs are in agreement with ur values, and differ nly in the furth decimal place; the differences are attributed t the fact that we use a different numerical ptimizatin rutine. Included in Table 1 are the asympttic standard errrs reprted in Newbld and Bs. Als shwn in the Table 1 are the crrespnding standard errrs btained frm B = 500 runs f the btstrap. These standard errrs are simply the square rt f B b=1 ( Θ ib Θ i ) 2 /(B 1), where Θ i,representstheith parameter, i =1,..., 5, and Θ i is the MLE f Θ i. The asympttic standard errrs listed in Table 1 are typically smaller than thse btained frm the btstrap. This result is the mst prnunced in the estimates f φ, σ w,andσ v,wherethe btstrapped standard errrs are abut 50% larger than the crrespnding asympttic value. Als, asympttic thery prescribes the use f nrmal thery when dealing with the parameter estimates. The btstrap, hwever, allws us t investigate the small sample distributin f the estimatrs and, hence, prvides mre insight int the data analysis. Fr example, Figure 1 shws the btstrap distributin f the estimatr f φ. This distributin is highly skewed with values cncentrated arund 0.8, but with a lng tail t the left. Sme quantiles 3

5 Table 1: Cmparisn f Asympttic Standard Errrs (SE) and Btstrapped Standard Errrs (B = 500). Asympttic Newbld & Bs Btstrap Parameter MLE SE SE SE φ α b σ w na σ v na Figure 1: Btstrap distributin, B = 500, f the estimatr f φ. f the btstrapped distributin f φ are 0.09 (2.5%), 0.03 (5%), 0.16 (10%), 0.87 (90%), 0.92 (95%), 0.94 (97.5%), and they can be used t btain cnfidence intervals. Fr example, a 90% cnfidence interval fr φ wuld be apprximated by (0.03, 0.92). This interval is rather wide, and we will interpret this after we discuss the results f the estimatin f σ w. Figure 2 shws the btstrap distributin f the estimatr f σ w. The distributin is cncentrated at tw lcatins, ne at apprximately σ w =0.15 and the ther at σ w =0. The cases in which σ w 0 crrespnd t deterministic state dynamics. When σ w =0and φ < 1, then β t b fr large t, s the apprximately 25% f the cases in which σ w 0 suggest a fixed state, r cnstant cefficient mdel. The cases in which σ w is away frm zer wuld suggest a truly stchastic regressin parameter. T investigate this matter further, Figure 3 shws the jint btstrapped estimates, ( φ, σ w ), fr nn-negative values f φ. The jint distributin suggests σ w > 0 crrespnds t φ 0. When φ = 0, the state dynamics are given by β t = b + w t. If, in additin, σ w is small 4

6 Figure 2: Btstrap distributin, B = 500, f the estimatr f σ w. phi sigma Figure 3: Jint btstrap distributin, B = 500, f the estimatrs f φ and σ w. Only the values crrespnding t φ 0areshwn. relative t b (as it appears t be in this case), the system is nearly deterministic; that is, β t b. Cnsidering these results, the btstrap analysis leads us t cnclude the dynamics f the data are best described in terms f a fixed, rather than stchastic, regressin effect. 5

7 If, hwever, we use the same mdel fr the entire data set presented in Newbld and Bs (that is, 110 quarters f three-mnth treasury bills and inflatin rate, cvering 1953:I t 1980:II), stchastic regressin appears t be apprpriate. In this case the estimates using Newtn-Raphsn with estimated standard errrs ( asympttic btstrap ) are: φ =0.896 ( ), α = ( ), b =1.090 ( ), σ w =0.117 ( ), σ v =1.191 ( ). We nte that the asympttic standard errr estimates are still t small, and the btstrapped distributin f φ is still markedly skewed. In particular, a 90% btstrap cnfidence interval fr φ is (.46,.92). 2.2 Stchastic Vlatility This prblem is smewhat different than the previus sectin in that it is nt a straight-frward applicatin f the algrithm. In this example, we cnsider the stchastic vlatility mdel due t Harvey, Ruiz and Shephard (1994). Let r t dente the return r grwth rate f a prcess f interest. Fr example, if s t is the value f a stck at time t, the return r relative gain f the stck is r t =ln(s t /s t 1 ). Typically, it is var(r t )=σ 2 t that is f interest. In the stchastic vlatility mdel, we mdel h t =lnσ 2 t as an AR(1), that is, h t+1 = φ 0 + φ 1 h t + w t, (12) where w t is white Gaussian nise with variance σ 2 w; this cmprises the state equatin. The bservatins are taken t be y t =lnr 2 t,andy t is related t the state via y t = α + h t + v t. (13) Tgether, (12) and (13) make up the stchastic vlatility mdel, where h t represents the unbserved vlatility f the prcess y t.ifv t was Gaussian white nise, (12) (13) wuld frm a Gaussian state space mdel, and we culd then use standard results t fit the mdel t data. Unfrtunately, y t =lnrt 2 is rarely nrmal, s ne typically assumes that v t =lnzt 2 where z t is standard Gaussian white nise. In this case, ln zt 2 is distributed as the lg f a chi-squared randm variable with ne degree f freedm. Kim, Shephard and Chib (1998) prpsed mdeling the lg f a chi-squared randm variable by a mixture f nrmals. Varius appraches t the fitting f stchastic vlatility mdels have been examined; these methds include a wide range f assumptins n the bservatinal nise prcess. A gd summary f the prpsed techniques, bth Bayesian (via MCMC) and nn-bayesian appraches (such as quasi-maximum likelihd estimatin and the EM algrithm), can be fund in Jacquier et al (1994), and Shephard (1996). Simulatin methds fr classical inference applied t stchastic vlatility mdels are discussed in Danielsn (1994) and Sandmann and Kpman (1998). In an effrt t keep matters simple, ur methd (see Shumway and Stffer, 2000, 4.10) f fitting stchastic vlatility mdels is t retain the Gaussian state equatin (12), but in the bservatin equatin (13), we cnsider v t t be white nise, and distributed as a mixture f tw nrmals, ne centered at zer. In particular, we write v t =(1 η t )z t0 + η t z t1, (14) 6

8 where η t is an iid Bernulli prcess, Pr{η t =0} = π 0, Pr{η t =1} = π 1, with π 0 + π 1 =1,and where z t0 iid N(0,σ 2 0 ), and z t1 iid N(µ 1,σ 2 1 ). The advantage f this mdel is that it is fairly easy t fit because it uses nrmality. The mdel specified by equatins (12) (14), and the crrespnding filter, are similar t thse presented in Peña and Guttman (1988), wh used the idea t btain a rbust Kalman filter, and, as previusly mentined, Kim, Shephard and Chib (1998). In additin, this technique is similar t technique discussed in Shumway and Stffer (2000, 4.8). In particular, the filtering equatins fr this mdel are: h t t+1 = φ 0 + φ 1 h t 1 Pt+1 t = φ 2 1 P t t 1 1 t + j=0 π tj K tj ɛ tj, (15) 1 + σw 2 π tj Ktj 2 Σ tj, (16) j=0 ɛ t0 = y t α ht t 1, (17) ɛ t1 = y t α ht t 1 µ 1, (18) Σ t0 = Pt t 1 + σ0 2, (19) Σ t1 = Pt t 1 + σ1 2, (20) K t0 = φ 1 Pt t 1 /Σ t0, (21) K t1 = φ 1 Pt t 1 /Σ t1. (22) T cmplete the filtering, we must be able t assess the prbabilities π t1 =Pr(η t =1 y 1,...,y t ), fr t =1,...,n;fcurse,π t0 =1 π t1.letf j (t t 1) dente the cnditinal density f y t given the past y 1,..., y t 1,andη t = j (j =0, 1). Then, π t1 = π 1 f 1 (t t 1) π 0 f 0 (t t 1) + π 1 f 1 (t t 1), (23) where we assume the distributin π j,frj =0, 1 has been specified apriri. If the investigatr has n reasn t prefer ne state ver anther the chice f unifrm prirs, π 1 =1/2, will suffice. Unfrtunately, it is cmputatinally difficult t btain the exact values f f j (t t 1); althugh we can give an explicit expressin f f j (t t 1), the actual cmputatin f the cnditinal density is prhibitive. A viable apprximatin, hwever, is t chse f j (t t 1) t be the nrmal density, + µ j,σ tj ), fr j =0, 1andµ 0 = 0; see Shumway and Stffer (2000, 4.8) fr details. The innvatins filter given in (15) (23) can be derived frm the Kalman filter by a simple cnditining argument. Fr example, t derive (15), we write N(h t 1 t E(h t+1 y 1,...,y t ) = 1 E(h t+1 y 1,...,y t,η t = j)pr(η t = j y 1,...,y t ) j=0 = 1 ( ) φ 0 + φ 1 ht t 1 + K tj ɛ tj π tj j=0 = 1 φ 0 + φ 1 ht t 1 + π tj K tj ɛ tj. j=0 7

9 Estimatin f the parameters, Θ = (φ 0,φ 1,σ0 2,µ 1,σ1 2,σ2 w), is accmplished via MLE based n the likelihd given by n 1 ln L Y (Θ) = ln π j f j (t t 1), (24) t=1 j=0 where the densities fr f j (t t 1) are apprximated by the nrmal densities previusly mentined. T perfrm the btstrap, we develp a vectr first-rder equatin, as was dne in (9). First, using (17) (18), and nting that y t = π t0 y t + π t1 y t,wemaywrite Cnsider the standardized innvatins and define the 2 1 vectr Als, define the 2 1 vectr y t = α + h t 1 t + π t0 ɛ t0 + π t1 (ɛ t1 + µ 1 ). (25) e tj =Σ 1/2 tj ɛ tj, j =0, 1, (26) e t = [ et0 e t1 [ h t ξ t = t+1 Cmbining (15) and (25) results in a vectr first-rder equatin fr ξ t given by where F = [ ] [ φ1 0, G t = 1 0 y t ]. ]. ξ t = Fξ t 1 + G t + H t e t, (27) φ 0 α + π t1 µ 1 ] [ πt0 K t0 Σ 1/2 t1, H t = π t0 Σ 1/2 t0 π t1 Σ 1/2 t1 t0 π t1 K t1 Σ 1/2 Hence, the steps in btstrapping fr this case are the same as steps 1 thrugh 5 previusly described, but with (11) replaced by the fllwing first-rder equatin: ξ t = F ( Θ)ξ t 1 + G t ( Θ; π t1 )+H t ( Θ; π t1 )e t, (28) where Θ =( φ 0, φ 1, σ 2 0, α, µ 1, σ 2 1, σ2 w) is the MLE f Θ, and π t1 is estimated via (23), replacing f 1 (t t 1) and f 0 (t t 1) by their respective estimated nrmal densities ( π t0 =1 π t1 ). T examine the efficacy f the btstrap fr the stchastic vlatility mdel, we generated n = 200 bservatins frm the fllwing stchastic vlatility mdel: h t =.95h t 1 + w t, (29) where w t is white Gaussian nise with variance σw 2 = 1. The bservatins were then generated as y t = h t + v t, (30) where the bservatinal white nise prcess, v t, is distributed as the lg f a chi-squared randm variable with ne degree f freedm. The density f v t is given by f v (x) = 1 { exp 1 } 2π 2 (ex x) <x<, (31) ]. 8

10 Figure 4: Simulated data, n = 200, frm the stchastic vlatility mdel (29) (30). and its mean and variance are 1.27 and π 2 /2, respectively; the density (31) is highly skewed with a lng tail n the left. The data are shwn in Figure 4. Then, we assumed the true errr distributin was unknwn t us, and we fit the mdel (12) (14) using the Gauss BFGS variable metric algrithm t maximize the likelihd. The results fr the state parameters are given in Table 2intheclumnsmarkedMLE and Asympttic SE. Next, we btstrapped the data, B = 500 times, using the incrrect mdel (12) (14) t assess the finite sample standard errrs (SE). The results are listed in Table 2 in the clumn marked Btstrap SE. Finally, using the crrect mdel, (29) (30), we simulated 500 prcesses, estimated the parameters based n the mdel (12) (14) als via a BFGS variable metric algrithm, and assessed the SEs f the estimates f the actual state parameters. These values are listed in Table 2 in the clumn labeled True SE. Table 2: Stchastic Vlatility Simulatin Results. State Actual Asympttic Btstrap True Parameter Value MLE SE SE SE φ σ w Based n 500 btstrapped samples. Based n 500 replicatins. In Table 2 we ntice that the btstap SE and the asympttic SE f φ are abut the same; als, bth estimates are slightly smaller than the true value. The interest here, hwever, is nt s much in the SEs, but in the actual sampling distributin f the estimates. T explre the finite sample distributin f the estimate f φ, Figure 5 shws the centered btstrap histgram: ( φ b φ), fr b =1,...,500 btstrapped replicatins [the bars are filled with lines f psitive slpe], the centered 9

11 true histgram: ( φ j φ), where φ j is the MLE btained n the j-th iteratin, fr j =1,...,500 Mnte Carl replicatins [the bars are filled with flat lines], and the centered asympttic nrmal distributin f ( φ φ) [apprpriately scaled fr cmparisn with the histgrams], superimpsed n eachther. Clearly, the btstrap distributin is clser t the true distributin than the estimated asympttic nrmal distributin; the btstrap distributin captures the psitive kurtsis (peakedness) and asymmetry f the true distributin. Figure 5: Sampling distributins f the estimate f φ; simulated data example: The centered btstrap histgram (lines with psitive slpe), the centered true histgram (flat lines), and the centered asympttic nrmal distributin. In an example using actual data, we cnsider the analysis f quarterly U.S. GNP frm 1947(1) t 2002(3), n = 223. The data are seasnally adjusted and were btained frm the Federal Reserve Bank f St. Luis ( The grwth rate is pltted in Figure 6 and appears t be a stable prcess. Analysis f the data indicates the grwth rate is an MA(2) [fr mre details f this part f the analysis, see Shumway and Stffer, 2000, 2.8], hwever, the residuals f that fit, which appear t be white, suggest that there is vlatility. Figure 7 shws the lg f the squared residuals, say y t, frm the MA(2) fit n the U.S. GNP series. The stchastic vlatility mdel (12) (14) was then fit t y t. Table 3 shws the MLEs f the mdel parameters alng with their asympttic SEs assuming the mdel is crrect. Als displayed in Table 3 are the means and SEs f B = 500 btstrapped samples. As in the simulatin, there is sme amunt f agreement between the asympttic values and the btstrapped values. Based n the previus simulatin, we wuld be mre prne t fcus n the actual sampling distributins, rather than assume nrmality. Fr example, Figure 8 cmpares the btstrap histgram and asympttic 10

12 Figure 6: U.S. GNP quarterly grwth rate. Figure 7: Lg f the squared residuals frm an MA(2) fit n GNP grwth rate. nrmal distributin f φ 1. In this case, as in the simulatin, the btstrap distributin exhibits psitive kurtsis and skewness which is missed by the assumptin f asympttic nrmality. Based n the simulatin, we wuld be prne t believe the results f the btstrap are fairly accurate. 11

13 Figure 8: Btstrap histgram and asympttic distributin f φ 1 fr the US GNP example. Table 3: Estimates and Their Asympttic and Btstrap Standard Errrs fr US GNP Example. Asympttic Btstrap Btstrap Parameter MLE SE Mean SE φ φ σ w α µ σ σ Based n 500 btstrapped samples. 3 Assessing the Finite Sample Distributin f Cnditinal Frecasts In this sectin we fcus n assessing the cnditinal frecast accuracy f time series mdels using a state space apprach and resampling methds. Our wrk is mtivated by the fllwing cnsideratins. First, the state space mdel prvides a cnvenient unifying representatin fr varius mdels, including ARMA(p, q) mdels. Secnd, the actual practice f frecasting invlves the predictin f a future pint based n an bserved sample path, thus cnditinal frecast errr assessment is f mst interest. Third, real-life applicatins invlving time series data are ften characterized by shrt data sets and lack f distributinal infrmatin. Asympttic thery prvides little help 12

14 here and ften there are n cmpelling reasns t assume Gaussian distributins apply. Finally, the utility and applicability already demnstrated by the btstrap fr predictin f AR prcesses suggests that it has much t ffer in the predictin f ther prcesses. Early applicatin f the btstrap t assess cnditinal frecast errrs can be fund in Findley (1986), Stine (1987), Thmbs and Schuchany (1990), Kabaila (1993) and McCullugh (1994, 1996). Interest in the evaluatin f cnfidence intervals fr cnditinal frecast errrs has led t methdlgical prblems because a backward, r reverse-time, set f residuals must be generated. Findley (1986) first discussed this prblem and Breidt, Davis and Dunsmuir (1992, 1995) ffered a slutin that is implemented in the wrk f McCullugh (1994, 1996). T date there is a well grunded methdlgy fr AR mdels and this wrk has established the utility f the btstrap. A similar state f affairs appears nt t exist fr ther time series mdels. We suspect this is due t the difficulty with which ne can identify mechanisms required t generate btstrap data sets, whether frwards r backwards in time. Fr AR mdels this is easily accmplished because the required initial, r terminal (in the case f cnditinal frecasts), cnditins are given in terms f the bserved series. With ther time series mdels this may nt be the case because the mdels require slutins f difference equatins invlving unbserved disturbances. The state space mdel and its related innvatins filter ffer a way arund this difficulty. It is wrthwhile, therefre, t investigate hw well this can be dne in practice. In 2, such a cmbinatin was f use in assessing parameter estimatin errr, and this naturally leads t the same questin being asked in relatin t cnditinal predictin errrs. We find that the btstrap is as useful in evaluating cnditinal frecast errrs as it has prven t be in assessing parameter estimatin errrs, particularly in a nn-gaussian envirnment. Our presentatin is based n the wrk f Wall and Stffer (2002). 3.1 Generating Reverse Time Datasets As seen in 2, the generatin f btstrap data sets in frward time is easy. Given an initial cnditin r prir, (11) is slved recursively fr t =1,...,n t prduce realizatins passing thrugh the given initial cnditin. Such cmputatins are all that is required in btaining btstrap estimates f parameter estimatin errr statistics r uncnditinal frecast errr statistics. The generatin f btstrap data sets fr assessing cnditinal frecast errrs is nt s straight frward because they must be generated backward and this requires a backward-time state space mdel. An early discussin f the prblems related t backward time mdels in assessing cnditinal frecast errrs is fund in Findley (1986). Further cnsideratin f the prblem is fund in Breidt, Davis and Dunsmuir (1992, 1995). This literature stresses the need t prperly cnstruct a set f backward residuals and Breidt, Davis and Dunsmuir (1992, 1995) prvide an algrithm fr this that slves the prblem fr AR(p) mdels. A similar result is needed fr state space mdels, but develpment f backward-time representatins has nt received much attentin in the literature. Ntable exceptins are the elegant presentatin fund in Caines (1988, Ch 4) and a derivatin in Aki (1989, Ch 5). Our wrk requires an extensin f their results t the time-varying case. The key system in generating btstrap data sets is the innvatins filter frm, (9); recall ξ t = F t ξ t 1 + Gu t + H t e t, (9) 13

15 where [ x t t+1 ξ t = y t ], F t = [ ] [ ] Φ 0 Υ, G =, H t = A t 0 Γ [ Kt Σ 1/2 ] t Σ 1/2. t We require a backward-time representatin f this system. All the prblems highlighted by Findley (1986) and Breidt, Davis and Dunsmuir (1992, 1995) appear here. Fr example, the first p rws f (9) cannt be slved backwards in time by simply expressing xt t 1 in terms f x t t+1. First, Φ is nt always invertible; e.g., MA(q) mdels. Secnd, even when Φ is invertible, Φ 1 has characteristic rts utside the unit circle whenever Φ has its characteristic rts inside the unit circle. This situatin is intlerable in generating reverse time trajectries because f the explsive nature f the slutins fr ξ t. In additin, we nw have a time-varying system. These difficulties are vercme by building n the methd fund in Caines (1988, pp ). Special attentin must be given t the way in which the time-varying matrices prpagate thrugh the derivatins and prper accunt must be taken f the effects f the knwn, r bserved input sequence u t. Fr ease, we will assume here that u t 0; the general case is presented in Wall and Stffer (2002). Applicatin f the symmetry f minimal splitting subspaces yields the fllwing reverse-time state space representatin fr t = n 1,n 2,...,1: r t =Φ r t+1 + B t x t 1 t C t e t where y t = N t r t+1 L t x t 1 t + M t e t B t = V 1 t Φ V 1 t+1 Φ C t = Φ Vt+1 1 tσ 1/2 t, D t = I Σ 1/2 t K tv t+1 1 tσ 1/2 t, L t = Σ 1/2 t C t A tv t B t, M t = Σ 1/2 t D t A t V t C t, N t = A t V t Φ +Σ 1 t K t, and V t+1 =ΦV t Φ + K t Σ 1 t K t. The reverse-time state vectr is r t. The backward recursin is initialized by r n = Vn 1xn 1 n. Details f the derivatin are given in Wall and Stffer (2002). The abve recursin specifies a three step prcedure fr the generatin f backward time data sets (written here fr u t 0): 1. Generate V t,b t,c t,d t,l t,m t and N t frwards in time, t =1,...,n, with initial cnditin V 1 = P 0 1 (32) 2. Fr given {e t ; 1 t n 1}, set x 1 = 0 and generate {x t ; 1 t n} frwards in time, t =1,...,n, via x t+1 =Φx t + K t Σ 1/2 t e t (33) 14

16 Figure 9: Reverse time realizatins f the ARMA(2, 1) prcess given in (36). 3. Set r n = r n = Vn 1xn 1 n and generate {y t ;1 t n} backwards in time, t = n 1,n 2...,1, viathereversetimestatespacemdel r t = Φ r t+1 + B tx t C te t (34) y t = N t r t+1 L t x t + M te t (35) This prcedure assumes ne already has drawn randmly, with replacement, frm the mdel estimated standardized residuals t btain a set f n 1 residuals dented {e t ; 1 t n 1}. The last residual is kept set at e n = e n in rder t ensure the cnditining requirement is met n ξ n ; that is, ξ n = ξ n. This requirement fllws frm the autregressive structure f (9); fr details, see 4.2. The creatin f an arbitrary number f btstrap data sets is accmplished by repeating the abve fr each set f btstrap residuals {e t ; 1 t n 1; e n = e n }. As an example, cnsider the univariate ARMA(p, q) prcess given by y t + a 1 y t a p y t p = v t + b 1 v t b q v t q where v t is an i.i.d. prcess with variance σv. 2 This prcess can be represented in state space frm, (1) (2), in varius ways. Fr example, let m =max{p, q}, letx t be an m-dimensinal state vectr, and write the state space cefficient matrices as a m a m 1. Φ = , a a a 1 15

17 A = [ ], Υ=0andΓ=0. The state nise prcess is defined by the w t = gv t where [ ] g = b m a m, b m 1 a m 1, b 3 a 3, b 2 a 2, b 1 a 1. If m>pthen a l =0frl>p,andifm>qthen b l =0frl>q.The variance-cvariance matrices are given by Q = σ 2 v gg, R = σ 2 v, S = σ2 v g. Figure 9 presents a sample f 100 reverse-time trajectries fr the Gaussian ARMA(2, 1) mdel y t =1.4y t y t 2 + v t +0.6v t 1, (36) with σ v =0.2 andn =49. The riginal, bserved sample is pltted with the bld line. 3.2 Cmputing Frecast Errrs via the Btstrap At this pint we assume we have n bservatins, y 1,...,y n, and we wish t frecast m time pints int the future. In additin, we have the MLEs f the mdel parameters Θ, say Θ, based n the data. The assciated standardized innvatin values are dented by {e t ( Θ); 1 t n}; nte,t avid any pssible cnfusin, we emphasize the dependence f the values n the parameters. Fr b = 1, 2,..., B (where B is the number f btstrap replicatins) we execute the fllwing six steps: 1. Cnstruct a sequence f n + m standardized residuals {e b t( Θ); 1 t n + m} via n + m 1 randm draws, with replacement, frm the standardized residuals {e t ( Θ); 1 t n}. This sequence is frmed as fllws: (i) use n 1 vectrs t frm {e b t( Θ); 1 t n 1}; (ii) fix e b n( Θ) = e n ( Θ); and (iii) use the remaining m vectrs t frm {e b t( Θ); n +1 t n + m}. 2. Generate data {y b t ( Θ); 1 t n 1} via the backward state space mdel (34) and (35) with Θ = Θ using the residuals {e b t ( Θ); 1 t n 1}. Set y b n ( Θ) = y n. 3. Generate data {y b t( Θ); n +1 t m + n} via the frward state space mdel (9) with Θ = Θ andwithx t 1;b t residuals e b t( Θ), fr n +1 t n + m. = xt t 1 ( Θ) and using the 4. Cmpute mdel parameter estimates Θ b via MLE using the data {y b t( Θ); 1 t n}. 16

18 5. Cmpute the btstrap cnditinal frecasts {y b t (Θb ); n +1 t m + n} via the frward time state space mdel (9) with Θ = Θ b,andwithx t 1;b t e b t = 0 fr n +1 t n + m. = xt t 1 (Θ b )and 6. Cmpute the btstrap cnditinal frecast errrs via: δ b l = yb n+l ( Θ) y b n+l (Θb ); 1 l m. The extent t which the btstrap captures the behavir f the actual frecast errrs derives frm the extent t which these errrs mimic the stchastic prcess δ l = y n+l (Θ) y n+l ( Θ); 1 l m. As an example, cnsider the univariate ARMA(1, 1) prcess given by y t =0.7y t 1 + v t +0.10v t 1 (37) where v t =0.2z t and z t is a mixture f 90% N(µ = 1/9,σ =.15) and 10% N(µ =1,σ =.15). T demnstrate the benefits f resampling, we will assume that we d nt knw the true distributin f v t and will act as if it was nrmal. The mdel is first-rder with Φ=[0.70] A = [1] and g =[0.80], in the ntatin f previus example. In this simulatin we use B =2, 000 and m = 4. The apprximate true distributin is then given by the relative frequency histgram f the bserved cnditinal frecast errrs. The results f the simulatin is summarized by tw sets f fur histgrams. One set (Figure 10) presents the apprximate true relative frequency histgrams fr each frecast lead time, while the ther set (Figure 11) presents the relative frequency histgrams btained frm applicatin f the btstrap. Superimpsed n each is the Gaussian density that fllws frm applicatin f the asympttic Gaussian thery. The simulatin uses a shrt data set with n =49temphasizetheefficacyf the btstrap when the use asympttics is questinable and where bias is a factr in the frecasts. Predictin intervals fllw immediately frm the data summarized in the histgrams. Althugh we chse t present nly the histgrams, the percentile, the bias-crrected (BC), and the accelerated bias-crrected (BC a ) methd all are applicable fr generating cnfidence intervals using the generated data (see Efrn, 1987). Figures 10 and 11 reveal the value f the btstrap. Indicatin f the mixture distributin is striking in bth the true and the btstrap; the bimdality and asymmetry are clearly evident. 3.3 Stchastic Regressin We nw illustrate the use f the btstrap in assessing frecast errrs in the data set analyzed in 2.1. Recall, the treasury bill interest rate is mdeled as being linearly related t quarterly inflatin as y t = α + β t z t + v t, 17

19 Figure 10: True frecast histgrams fr the ARMA(1, 1) prcess given in (37). Figure 11: Btstrap frecast histgrams fr the ARMA(1, 1) prcess given in (37). where α is a fixed cnstant, β t is a stchastic regressin cefficient, and v t is white nise with variance σv 2. The stchastic regressin term, which cmprises the state variable, is specified by a first-rder autregressin, (β t b) =φ(β t 1 b)+w t, 18

20 Figure 12: Dymanic behavir f the quantiles f y b n+l ( Θ) [upper left panel], the quantiles f y b n+l (Θb ) [upper right panel], and the quantiles f the btstrap cnditinal frecast errrs y b n+l ( Θ) y b n+l (Θb ), fr l =1, 2, 3, 4 [lwer left panel]. The y t series [lwer right panel] as a bld line and the envelpe f the backward data series as fine lines abve and belw the bserved sample in the stchastic regressin example. where b is a cnstant, and w t is white nise with variance σ 2 w. The nise prcesses, v t and w t,are assumed t be uncrrelated. The mdel parameter vectr cntains five elements, Θ = (φ, α, b, σ w,σ v ) and is estimated via Gaussian quasi-maximum likelihd using data frm the first quarter f 1967 thrugh the secnd quarter f 1979 (49 bservatins). The MLEs and their estimated standard errrs (in parentheses) were: φ =0.898 (0.101) α = (1.457) b =1.195 (0.278) σ w =0.092 (0.049) σ v =1.287 (0.197) Amng the many frecast errr assessment questins that can be asked cncerning this mdel are nes cncerning the prperties f the cnditinal frecast errr distributin assuming that we knw the future values f the inflatin rate. In particular, is a Gaussian assumptin warranted when assume we knw the actual future values f z t? Such questins may arise within the cntext f a ratinal expectatins framewrk wherein ecnmic agents are assumed s well infrmed that they knw the inflatin rate. The btstrap, cupled with ur methdlgy here, can shed sme light n just such a questin as this. Figure 12 depicts the btstrap results with B = The upper left panel presents the dynamic behavir f the quantiles (specifically, 2.5%, 5%, 16%, 50%, 84%, 95%, 97.5%) f y b n+l ( Θ) and the upper right panel presents the quantiles f y b n+l (Θb ). Given the significant variability in the upper right panel, it is clear that the variability due t the additive disturbances (upper left 19

21 Figure 13: Histgrams f fur cnditinal frecast errrs, B = 2000, in the stchastic regressin example. Figure 14: Histgrams f fur cnditinal frecast errrs, B = 10, 000, in the stchastic regressin example. panel) is nt the dminant factr in the frecast uncertainty that it is s ften assumed t be. The lwer left panel the depicts dynamic behavir f the quantiles f the btstrap cnditinal frecast errrs y b n+l ( Θ) y b n+l (Θb ), fr l =1, 2, 3, 4. The lwer right panel plts the y t series as a bld line 20

22 and the envelpe f the backward data series as fine lines abve and belw the bserved sample. We find the backward generated series t be highly representative f the stchastic prperties f the bserved series. Figure 13 presents histgrams f the cnditinal frecast errrs, y b n+l ( Θ) y b n+l (Θb ), fr l = 1, 2, 3, 4, when B = 2000 and Figure 14 presents the histgrams when B =10, 000. Each picture gives indicatin f the prblems in assuming that the asympttic thery applies. Negative bias is indicated and t-tests reject zer means fr l =2, 3, 4, in bth btstrap experiments. A Klmgrv-Smirnv test rejects the asympttic Gaussian distributin (which are als displayed in the figures) fr all frecast lead times fr bth values f B. It appears little is gained in extending the btstrap replicatins beynd B = 2000, ther than the mre smth appearance f the histgrams. 4 Discussin The state space mdel prvides a cnvenient unifying representatin fr varius time dmain mdels. This article demnstrates the utility f resampling the innvatins f time dmain mdels via state space mdels and the Kalman (innvatins) filter. We have based ur presentatin primarily n the material in tw articles, Stffer and Wall (1991) and Wall and Stffer (2002). In Stffer and Wall (1991) we develped a resampling scheme t assess the finite sample distributin f parameter estimates fr general time dmain mdels. This algrithm uses the elegance f the state space mdel in innvatins frm t cnstruct a simple resampling scheme. The key pint is that while under general cnditins, the MLEs f the mdel parameters are cnsistent and asympttically nrmal, time series data are ften f shrt r mderate length s that the use f asympttics may lead t wrng cnclusins. Mrever, it is well knwn that prblems ccur if the parameters are near the bundary f the parameter space. We have prvided additinal examples here that emphasize the usefulness f the algrithm. We have als explained, heuristically, why the resampling scheme is asympttically crrect under apprpriate cnditins. We have als discussed cnditinal frecast accuracy f time dmain mdels using a state space apprach and resampling methds that was first presented in Wall and Stffer (2002). Applicatins invlving time series data are ften characterized by shrt data sets and lack f distributinal infrmatin; asympttic thery prvides little help here and frequently there are n cmpelling reasns t assume Gaussian distributins apply. Interest in the evaluatin f cnfidence intervals fr cnditinal frecast errrs in AR mdels led t methdlgical prblems because a backward, r reverse-time, set f residuals must be generated. This prblem was eventually slved and there is nw a well grunded methdlgy fr AR mdels. Researchers were cnfined t AR mdels because the required initial, r terminal (in the case f cnditinal frecasts), cnditins are given in terms f the bserved series. With ther time series mdels this may nt be the case because the mdels require slutins f difference equatins invlving unbserved disturbances. The state space mdel and its related innvatins filter ffered a way arund this difficulty. We have exhibited a reversetime state space in innvatins frm. We have presented additinal examples here that demnstrate resampling as useful in evaluating cnditinal frecast errrs as it has prven t be in assessing parameter estimatin errrs, particularly in a nn-gaussian envirnment. In the Appendix, we explain, heuristically, why resampling wrks in large samples. 21

23 Appendix: Large Sample Heuristics In 2, resampling techniques were used t determine the finite sample distributins f the parameter estimates when the use f asympttics was questinable. In 3, we used resampling t assess the finite sample distributins f the frecast errrs. The extent t which resampling the innvatins des what it is suppsed t d can be measured in varius ways. In the finite sample case, we can perfrm simulatins where the true distributins are knwn and cmpare the btstrap results t the knwn results. If the btstrap wrks well in simulatins, we may feel cnfident that the btstrap will wrk well in similar situatins, but, f curse, we have n guarantee that it wrks in general. In this way, the examples in 2 and 3 help demnstrate the validity f the resampling prcedures discussed in thse sectins. Anther apprach is t ask if the btstrap will give the crrect asympttic answer. That is, if we have an infinite amunt f data and can resample an infinite amunt f times, d we get the crrect asympttic distributin (typically, we require asympttic nrmality). If the answer is n, we can assume that resampling will nt wrk with small samples. If the answer is yes, we can nly hpe that resampling will wrk with small samples, but again, we have n guarantee. Fr state space mdels, hw well the resampling techniques perfrm in finite samples hinges n at least three things. First, the techniques are cnditinal n the data, s the success f the resampling depends n hw typical the data set is fr the particular mdel. Secnd, we assume the mdel is crrect (at least apprximately); if the prpsed mdel is far frm the truth, the results f the resampling will als be incrrect. Finally, assuming the data set is typical and the mdel is crrect, the success f the resampling depends n hw clse the empirical distributin f the innvatins is t the actual distributin f the innvatins. We are guaranteed such clseness in large samples if the innvatins are stable and mixing in the sense f Gastwirth and Rubin (1975). Sectin 2 Heuristics Stffer and Wall (1991) established the asympttic justificatin f the prcedure presented in 2 under general cnditins (including the case where the prcess is nn-gaussian). T keep matters simple, we assume here that the state space mdel, (1) (2) with A t A, is Gaussian, bservable and cntrllable, and the eigenvalues f Φ are within the unit circle. We dente the true parameters by Θ 0, and we assume the dimensin f Θ 0 is the dimensin f the parameter space. Let Θ n be the cnsistent estimatr f Θ 0 btained by maximizing the Gaussian innvatins likelihd, L Y (Θ), given in (10). Then, under general cnditins (n ), ) n ( Θn Θ 0 AN [ 0, I n (Θ 0 ) 1], where I n (Θ) is the infrmatin matrix given by I n (Θ) = n 1 E [ 2 ln L Y (Θ) / Θ Θ ]. Precise details and the prf f this result are given in Caines (1988, Chapter 7) and in Hannan and Deistler (1988, Chapter 4). Let Θ n dente the parameter estimates btained frm the resampling prcedure f 2. Let B n be the number f btstrap replicatins and, fr ease, we take B n = n. Then, Stffer and Wall (1991) established that, under certain regularity cnditins (n ), n ( Θ n Θ ) [ n AN 0, In( Θ n ) 1], where I n(θ) is the infrmatin matrix given by I n(θ) = n 1 E [ 2 ln L Y (Θ) / Θ Θ ], and E dentes expectatin with respect t the empirical distributin f the innvatins. It was then shwn that I n (Θ 0 ) I n ( Θ n ) 0 (38) 22

24 almst surely, as n ; hence, the resampling prcedure is asympttically crrect. It is infrmative t examine, at least partially, why (38) hlds. Let Z ta Z ta (Θ) = (e t e t )/ θ a where e t e t (Θ) is the standardized innvatin, (8), and θ a is the a-th cmpnent f Θ. Similarly, let Zta Z ta (Θ) = (e t e t )/ θ a,wheree t e t (Θ) is the resampled standardized innvatin. The (a, b)-th element f I n (Θ 0 )is n 1 n t=1 whereas the (a, b)-th element f I n ( Θ n )is The terms in (40) are n 1 n t=1 E (Z ta) =n 1 {E(Z ta Z tb ) E(Z ta )E(Z tb )} Θ=Θ0 (39) {E (Z taz tb) E (Z ta)e (Z tb)} Θ= Θ n. (40) n j=1 n Z ja and E (ZtaZ tb) =n 1 Z ja Z jb. (41) Hence, (39) cntains ppulatin mments, whereas (40) cntains the crrespnding sample mments. It shuld be clear that under apprpriate cnditins, (39) and (40) are asympttically (n ) equivalent. Details f these results can be fund in Stffer and Wall (1991, Appendix). Sectin 3 Heuristics As in the previus part, t keep matters simple, we assume the state space mdel (1) (2), with A t A, is bservable and cntrllable, and the eigenvalues f Φ are within the unit circle; these assumptins ensure the asympttic stability f the filter. We assume that we have N bservatins, {y n N+1,...,y n } available, and that N is large. We let Θ N dente the (assumed cnsistent as N ) Gaussian MLE f Θ, and let Θ N dente a btstrap parameter estimate. Fr ne-step-ahead frecasting the mdel specifies that the prcess ξ t, which we assume is in steady-state at time n +1,isgivenby ξ n+1 = F (Θ)ξ n + G(Θ)u n + H(Θ)e n+1 (42) where and F = [ ] Φ 0, G = A 0 [ x t t+1 ξ t = y t j=1 ], (43) [ ] Υ, H = Γ [ KΣ 1/2 K and Σ represent the steady-state gain and innvatin variance-cvariance matrices, respectively. Recall that {u t } is a fixed and knwn input prcess. Fr cnvenience, we have drpped the parameter frm the ntatin when representing a filtered value that depends n Θ. Fr example, in (42) we wrte ξ t ξ t (Θ) and e t e t (Θ). The prcess e t is the standardized, steady-state innvatin sequence s that E{e t } =0andE{e t e t} = I q. The ne-step-ahead cnditinal frecast estimate is given by Σ 1/2 ξ ξ ξ n+1 = F ( Θ N ) ξ ξ ξ n + G( Θ N )u n, (44) where, in keeping cnsistent with the ntatin, we have written ξ ξ ξ n ξ n ( Θ N ). The cnditinal frecast estimate is labeled with a tilde. Watanabe (1985) shwed that, under the assumed cnditins and ntatin, ] ; 23

25 x n n+1( Θ N )=x n n+1(θ) + p (1) (N ), and cnsequently, we write ξ ξ ξ n = ξ n + p (1), nting that the final q elements f ξ ξ ξ n and ξ n are identical. Hence, the cnditinal predictin errr can be written as N ξ n+1 ξ ξ ξ n+1 = [F (Θ) F ( Θ N )] ξ ξ ξ n ++F (Θ) p (1) + [G(Θ) G( Θ N )]u n + H(Θ)e n+1 (45) Frm (45) we see the tw surces f variatin, namely the variatin due t estimating the parameter Θ by Θ N, and the variatin due t the predicting the innvatin value e n+1 by zer. In the cnditinal btstrap prcedure, we mimic (42) and btain a pseud bservatin ξ n+1 = F ( Θ N ) ξ ξ ξ n + G( Θ N )u n + H( Θ N )e n+1, (46) wherewehld ξ ξ ξ n fixed thrughut the resampling prcedure. Nte that because the filter is in steady-state, the data, {y n N+1,...,y n }, cmpletely determine Θ N and cnsequently ξ ξ ξ n. Fr finite sample lengths, the data and the initial cnditins determine Θ N. As a practical matter, if precise initial cnditins are unknwn, ne can drp the first few data pints frm the estimatin f Θ s that changing the initial state cnditins des nt change Θ N nr ξ ξ ξ n. We remark that while the data cmpletely determine ξ ξ ξ n, the reverse is nt true; that is, fixing ξ ξ ξ n in n way fixes the entire data sequence {y n N+1,...,y n }. Fr example, in the AR(1) case, fixing ξ ξ ξ n is equivalent t fixing y n nly. In additin, e n+1 is a randm draw frm the empirical distributin f the standardized, steady-state innvatins. Under the mixing cnditins f Gastwirth and Rubin (1975), the empirical distributin f the standardized, steady-state innvatins cnverges weakly (N ) t the standardized, steady-state innvatin distributin. T mimic the frecast in (44), the btstrap estimated cnditinal frecast is given by which yields the btstrapped cnditinal frecast errr ξ ξ ξ n+1 = F ( Θ N) ξ ξ ξ n + G( Θ N )u n, (47) N ξ n+1 ξ ξ ξ n+1 = [F ( Θ N ) F ( Θ N )] ξ ξ ξ n +[G( Θ N ) G( Θ N )]u n + H( Θ N )e n+1. (48) Cmparisn f (45) and (48) shws why, in finite samples, the btstrap wrks; that is, (48) is a sample-based imitatin f (45). Letting N in (45), while hlding ξ ξ ξ n fixed, we have that, if Θ N p Θ, then N H(Θ)e where e is a randm vectr that is distributed accrding t the steady-state standardized innvatin distributin ( dentes weak cnvergence). In additin, if cnditinal n the data, Θ N Θ N p 0, then N H(Θ)e as N. Extending these results t m-step-ahead frecasts fllws easily by inductin. Stffer and Wall (1991) established cnditins under which Θ N Θ N p 0asN when the frward btstrapped samples are used. It remains t determine the cnditins under which this result hlds when the backward btstrap data are used. Acknwledgments: The wrk f D.S. Stffer was supprted, in part, by the Natinal Science Fundatin under Grant N. DMS