Strong Approximation for the Sums of Squares of Augmented GARCH Sequences

Transcription

1 Strong Approximation for the Sums of Squares of Augmented GARCH Sequences Alexander Aue 1 István Berkes Lajos Horváth 3 Abstract: We study so called augmented GARCH sequences, which include many submodels of considerable interest, such as polynomial and exponential GARCH. To model the returns of speculative assets, it is particularly important to understand the behaviour of the squares of the observations. The main aim of this paper is to present a strong approximation for the sum of the squares. This will be achieved by an approximation of the volatility sequence with a sequence of blockwise independent random variables. Furthermore, we derive a necessary and sufficient condition for the existence of a unique strictly) stationary solution of the general augmented GARCH equations. Also, necessary and sufficient conditions for the finiteness of moments are provided. AMS 000 Subject Classification: Primary 60F15, Secondary 6M10. Keywords and Phrases: augmented GARCH processes, stationary solutions, strong approximation, moments, partial sums 1 Department of Mathematics, University of Utah, 155 South 1440 East, Salt Lake City, UT , USA, aue@math.utah.edu Institut für Statistik, Technische Universität Graz, Steyrergasse 17/IV, A 8010 Graz, Austria, berkes@stat.tu-graz.ac.at 3 Department of Mathematics, University of Utah, 155 South 1440 East, Salt Lake City, UT , USA, horvath@math.utah.edu Corresponding author: A. Aue, phone: , fax: Research partially supported by NATO grant PST.EAP.CLG and NSF OTKA grant INT 036

2 1 Introduction Starting with the fundamental paper of Engle 198), autoregressive conditionally heteroskedastic ARCH processes and their generalizations have played a central role in finance and macroeconomics. One important class of these generalizations can be subsumed under the notion of augmented GARCH1,1) processes introduced by Duan 1997). He studied random variables {y k : < k < } satisfying the equations 1.1) and 1.) y k = σ k ε k Λσ k ) = cε k 1)Λσ k 1 ) + gε k 1), where Λx), cx) and gx) are real valued functions. To solve for σk, the definitions in 1.1) and 1.) require that 1.3) Λ 1 x) exists. Throughout the paper we assume that 1.4) {ε k : < k < } is a sequence of independent, identically distributed random variables. We start by giving a variety of examples which are included in the framework of the augmented GARCH model. Since all submodels must satisfy 1.1), only the corresponding specific equations 1.) are stated. Moreover, it can be seen that 1.3) always holds true. Example 1.1 GARCH) Bollerslev 1986) introduced the process σ k = ω + βσ k 1 + αy k 1 = ω + [ β + αεk 1] σ k 1. Example 1. AGARCH) This asymmetric model for volatility was defined by Ding, Granger and Engle 1993) as σk = ω + βσk 1 + α y k 1 µy k 1 ) = ω + [ β + α ε k 1 µε k 1 ) ] σk 1. Example 1.3 VGARCH) Engle and Ng 1993) used the equations σk = ω + βσ k 1 + αε k 1 µ) to study the impact of news on volatility.

3 Example 1.4 NGARCH) This nonlinear asymmetric model was also introduced by Engle and Ng 1993): σk = ω + βσk 1 + αε k 1 µ) σk 1. Example 1.5 GJR GARCH) The model of Glosten, Jagannathan and Runke 1993) is given by σk = ω + βσk 1 + α 1yk 1 I{y k 1 < 0} + α yk 1 I{y k 1 0} = ω + [ β + α 1 ε k 1 I{ε k 1 < 0} + α ε k 1 I{ε k 1 0} ] σk 1. Example 1.6 TSGARCH) This is a modification of Example 1.1 studied by Taylor 1986). σ k = ω + βσ k 1 + α 1 y k 1 = ω + [β + α 1 ε k 1 ] σ k 1. Example 1.7 PGARCH) Motivated by a Box Cox transformation of the conditional variance, Carrasco and Chen 00) extensively studied the properties of the process satisfying σ δ k = ω + βσ δ k 1 + α y k 1 δ = ω + [ β + α ε k 1 δ] σk 1 δ. Example 1.8 TGARCH) Example 1.5 is a nonsymmetric version of the GARCH process of Example 1.1. Similarly, a nonsymmetric version of Example 1.7 is σk δ = ω + βσk 1 δ + α 1 y k 1 δ I{y k 1 < 0} + α y k 1 δ I{y k 1 0} = ω + [ β + α 1 ε k 1 δ I{ε k 1 < 0} + α ε k 1 δ I{ε k 1 0} ] σk 1 δ. The special case of δ = 1 was proposed by Taylor 1986) and Schwert 1989) and includes the threshold model of Zakoian 1994). In Examples , Λx) = x δ, so these models are usually referred to as polynomial GARCH. Next, we provide two examples for exponential GARCH. Example 1.9 MGARCH) The multiplicative model of Geweke 1986) is defined as log σ k = ω + β log σ k 1 + α log y k 1 = ω + α + β) log σ k 1 + α log ε k 1. 3

4 Example 1.10 EGARCH) The exponential GARCH model of Nelson 1991) is log σ k = ω + β log σ k 1 + α 1ε k 1 + α ε k 1. Straumann and Mikosch 005) obtained a necessary and sufficient condition for the existence and uniqueness of the solution of the recursive equations in Examples 1.1, 1. and Their method is based on the theory of iterated functions. Carrasco and Chen 00) studied the existence of solutions in the general framework of 1.1) and 1.). Since they were also interested in the mixing properties of the sequences {y k } and {σ k}, they assumed that c0) < 1, E cε 0 ) < and E gε 0 ) <. We show that these conditions can be weakened to logarithmic moment conditions. The verification of the mixing property of solutions to 1.1) and 1.) is based on the theory of hidden Markov models and geometric ergodicity, and it is also assumed that ε 0 has a continuous density which is positive on the whole real line. The exponential mixing proved by Carrasco and Chen 00) yields the weak convergence as well as the approximation of partial sums of y i with Brownian motion. However, for these results to hold true it should not be required to assume either the existence or smoothness of the density of ε 0. The main aim of this paper is to provide a general method on how to obtain strong invariance principles for partial sums of GARCH type sequences. To this end, we show that {σk } can be approximated with a sequence of random variables { σ k } which are defined in a way that σ i and σ j are independent if the distance i j is appropriately large. This is a generalization of the observation in Berkes and Horváth 001) that the volatility in GARCHp, q) models can be approximated with random variables which are blockwise independent. It is also known that the general theory of mixing provides a poor bound for the rate of convergence to Brownian motion. We connect this rate of convergence to the finiteness of moments of ε 0. The general blocking method used throughout the proofs yields strong invariance for the partial sums of the random variables {yk Ey k } with very sharp rates. Our method can, however, also be used to obtain strong approximations for other functions of y 1,...,y n. Additionally, while we are focusing on processes of order 1,1) for the sake of presentation, adaptations of our methods work also for sequences of arbitrary order p, q). Strong approximations of partial sums play an important role in, for instance, deriving results in asymptotic statistics. So, it has been pointed out by Horváth and Steinebach 000) that limit results for so called CUSUM and MOSUM type test procedures, which are used to detect mean and variance changes, can be based on strong invariance principles. Theorem.4, in particular, might be used to test for the stability of the observed volatilities. In addition, strong approximations can also be used to develope sequential 4

5 monitoring procedures cf. Berkes et al. 004), Aue and Horváth 004), Aue et al. 005), and Horváth et al. 005)). The paper is organized as follows. First, we discuss the existence of a unique stationary solution of the augmented GARCH model 1.1) and 1.). We also find conditions for the existence of the moments of Λσ0 ) cf. Theorems.1.3 below). These results will be utilized to provide strong approximations for the partial sums of {yk} in Theorem.4. The proofs of Theorems.1.3 are given in Section 4, while the strong approximation of Theorem.4 is verified in Section 5. In Section 3, we deal with applications of Theorem.4 in the field of change point analysis. Main results Solving 1.1) and 1.) backwards, we get that, for any N 1,.1) Λσk) = Λσk N) cε k i ) + gε k i ) cε k j ). 1 i N 1 i N 1 j i 1 This suggests that the general solution of 1.1) and 1.) is given by.) Λσk ) = gε k i ) cε k j ). 1 j i 1 Our first result gives a necessary and sufficient condition for the existence of the random variable X =.3) gε i ) cε j ). 1 j i 1 Let log + x = logmax{x, 1}). Theorem.1 We assume that 1.1) 1.4) hold,.4) i) If.5) E log + gε 0 ) < and E log + cε 0 ) <. E log cε 0 ) < 0, then the sum in.3) is absolutely convergent with probability one. ii) If.6) P {gε 0 ) = 0} < 1 and the sum in.3) is absolutely convergent with probability one, then.5) holds. 5

6 We note that in.5) we allow that E log cε 0 ) =. The result bears some resemblence with Theorem 1.3 in Bougerol and Picard 199), who concentrate, however, on positive processes and put more emphasis on the problem of irreducibility. The proof of Theorem.1 is given in Section 4. Since under stationarity Λσk) = D X, the next result gives conditions for the existence of E X ν as well as of E Λσk ) ν. Theorem. We assume that 1.1) 1.4),.5) hold, ν > 0 and.7) i) If.8) then.9) E gε 0 ) ν <. E cε 0 ) ν < 1, E X ν <. ii) If.6) and.9) are satisfied,.10) cε 0 ) 0 and gε 0 ) 0, then.8) holds. We note that in Examples the left hand side of the equations defining Λσk ) is always positive, so the right hand side of those equations must also be positive. This requirement restricts the values of possible parameter choices such that.10) always holds in these examples. We also point out that Theorem. is an extension of the results in Nelson 1990) on the existence of the moments of a GARCH1,1) sequence. The next result shows that.) is the only solution of 1.1) and 1.). We say that a strictly stationary solution of 1.1) and 1.) is nonanticipative if, for any k, the random variable y k is independent of {ε i : i > k}. According to Brandt 1986), the strictly stationary solution of 1.1) and 1.) in part i) of the following theorem is always nonanticipative. The same is true under the assumptions of part ii) of the same theorem. Theorem.3 We assume that 1.1) 1.4) and.4) are satisfied. i) If.5) holds, then.) is the only stationary solution of 1.1) and 1.). ii) If in addition.6),.10) hold and 1.1), 1.) has a stationary, nonnegative solution, then.5) must be satisfied. 6

7 Theorem.3 gives, in particular, a necessary and sufficient condition for the existence of polynomial GARCH1,1) and thus generalizes the result of Nelson 1990). The special case of EGARCH introduced in Example 1.10 is also covered by Straumann and Mikosch 005). Here, stationarity can be characterized as follows. If β < 1 and E log + α 1 ε 0 + α ε 0 <, then there is a unique stationary, nonanticipative process satisfying the equation in Example If β > 0 and P {α 1 ε 0 +α ε 0 0} = 1, then β < 1 is necessary and sufficient for the existence of a stationary solution of the EGARCH equations. If we allow anticipative solutions, then 1.1) and 1.) might have a solution even if.5) fails, i.e. if E log cε 0 ) 0 cf. Aue and Horváth 005)). Now, we consider approximations for Sk) = 1 i k y i Ey i ). Towards this end, we need that Λ 1 x) is a smooth function. We assume that.11) Λσ0 ) ω with some ω > 0, Λ exists and is nonnegative, and there are C, γ such that.1) 1 Λ Λ 1 x)) Cxγ for all x ω. Assumptions.11) and.1) hold for the processes in Examples , but fail for the remaining Examples 1.9 and Thus, more restrictive moment conditions are needed in the exponential GARCH case cf. Theorem.4ii) below). Set.13) σ = Var y0 + Covy0, yi ). Theorem.4 We assume that 1.1) 1.4),.5) hold, σ > 0,.14) E ε 0 8+δ < with some δ > 0. i) If.11) and.1) are satisfied and.7) holds for some ν > 41 + max{0, γ}), then there is a Wiener process {Wt) : t 0} such that.15) yi Eyi ) σwn) = o n 5/1+ε) a.s. 1 i n for any ε > 0. ii) If Λx) = log x, we assume that E expt gǫ 0 ) ) exists for some t > 4 and.16) there is 0 < c < 1 such that cx) < c, then.15) holds. 7

8 The method of the proof of Theorem.4 can be used to establish strong approximations for the sums of functionals of the y i, too. For example, approximations can be derived for 1 i ny i Ey i ), 1 i ny i y i+k E[y i y i+k ]) and 1 i nyi yi+k E[yi yi+k]), where k is a fixed integer. Examples of strong approximations for dependent sequences other than augmented GARCH processes may be found in Eberlein 1986) and Kuelbs and Philipp 1980). While we are using a blocking argument, the methods in the afore mentioned papers are based on examining conditional expectations and variances, and mixing properties, respectively. 3 Applications In this section, we illustrate the usefulness of Theorem.4 with some applications coming from change point analysis. One of the main concerns in econometrics is to decide whether the volatility of the underlying variables is stable over time or if it changes in the observation period. While, in general, this allows to include larger classes of random processes, we will focus on augmented GARCH sequences here. To accomodate the setting, we assume that the volatilities {σ k } follow a common pattern until an unknown time k, called the change point. After k a different regime takes over. It will be demonstrated in the following that, using Theorem.4, some of the basic detection methods can be applied to the observed volatilities {y k }. Application 3.1 The popular CUSUM procedure is based on the test statistics T n = 1 max n σ yi k y i 1 k n 1 i k n. 1 i n Under the assumption of no change in the volatility it holds 3.17) T n D sup Bt), 0 t 1 where {Bt) : 0 t 1} denotes a Brownian bridge. Theorem.4 not only gives an alternative way to prove 3.17), but also provides the strong upper bound On 1/1+ε ) for the rate of convergence. Application 3. A modification of Application 3.1 is given by the so called MOSUM procedure. Therein, MOSUM is the abbreviation for moving sum.) Its basic properties have been collected in Csörgő and Horváth 1997, p. 181). In our case, the test statistic becomes V n = 1 an) σ max 1 k n an) k<i k+an) 8 y i an) n yi. 1 i n

9 Let, in addition to the assumptions of Theorem.4, the conditions 3.18) n 5/1+ε logn/an)) an) = O1) for some τ > 0 and an)/n 0, as n, be satisfied. Then, we obtain the following extreme value asymptotic for V n : 3.19) for all t, where Ax) = { lim P A n ) )} n n V n t + D = exp e t) an) an) log x and Dx) = log x + 1 log log x 1 log π. We shall give a proof of 3.19) now. Indeed, on using Theorem.4, we obtain V n = 1 an) max Wk + an)) Wk) an) 1 k n an) n Wn) 3.0) + O n5/1+ε P an) as n. The moduli of continuity of the Wiener process cf. Csörgő and Révész 1981, p. 30) yield, as n, 3.1) 1 an) = max Wk + an)) Wk) 1 k n an) 1 an) ) log n sup Wt + an)) Wt) + O P 0 t n an) an) By the scale transformation we obtain for the functional of the Wiener process on the right hand side of the latter equation 3.) 1 an) sup Wt + an)) Wt) = D 0 t n an) sup Ws + 1) Ws). 0 s n/an) 1 Hence, Theorem 7..4 of Révész 1990, p. 7) gives the following extreme value asymptotic for the Wiener process. It holds, { ) n 3.3) lim P A n an) sup Ws + 1) Ws) t + D 0 s n/an) 1 On combining 3.0) 3.) with condition 3.18), we arrive at A ) n V n = D A an) ) n an) )} n = exp e t). an) sup Ws + 1) Ws) + o P 1), 0 s n/an) 1 9

10 so that 3.19) follows from 3.3). Application 3.3 It has been pointed out by Csörgő and Horváth 1997) that weighted versions of the CUSUM statistics have a better power for detecting changes that occur either very early or very late in the observation period. Therefore, T n from Application 3.1 can be transformed to the weighted statistics T n,α = 1 ) n α max n σ yi k 1 k n k 1 i k n yi, 1 i n where 0 α < 1/. Under the assumptions of Theorem.4 it holds 3.4) T n,α D Bt) sup 0<t<1 t α n ), where {Bt) : 0 t 1} is a Brownian bridge. Theorem.4 and the moduli of continuity of the Wiener process cf. Csörgő and Révész 1981, p. 30)) imply that 3.5) yi Ey 0 ) σwt) = o t 5/1+ε) a.s. 1 i t as t. Hence, 1 1 σwnt) sup n 1/n t 1 t α i 1 i nty Ey 0 ) 1 nt) 5/1+ε ) O ) P n 1/1+ε if α 5/1 + ε, = O P n sup = 1/n t 1 t α O ) P n α 1/ if 5/1 + ε < α < 1/, giving that T D 1 n,α = sup 1/n t 1 t Wt) tw1) + o P1). α By the almost sure continuity of t α Wt), we obtain sup 1/n t 1 1 Wt) tw1) sup tα 0<t 1 1 Wt) tw1) t a.s. α Since {Wt) tw1) : 0 t 1} and {Bt) : 0 t 1} have the same distribution 3.4) holds. Application 3.4 Instead of the weighted sup norm, also weighted L functionals can be used. Let Z n = y n σ i t 0 t 1 i nt 1 i n 10 yi dt.

11 Under the conditions of Theorem.4 we have that 3.6) Z n 1 D 0 B t) dt t n ), where {Bt) : 0 t 1} denotes a Brownian bridge. Using 3.5), we conclude Z n = 1 n t 0 1 t Wnt) twn)) dt + O P n 1/1+ε ). So, with Bt) = n 1/ Wnt) twn)), we immediately get 3.6). For on line monitoring procedures designed to detect changes in volatility confer Aue et al. 005). All applications contain the parameter σ, which is in general unknown and so it has to be estimated from the sample data y 1,...,y n. This can be achieved easily by imposing the method developed by Lo 1991). For further results concerning the estimation of the variance of sums of dependent random variables we refer to Giraitis et al. 003) and Berkes et al. 005). 4 Proofs of Theorems.1.3 Proof of Theorem.1. Part i) is an immediate consequence of Brandt 1986). The first step in the proof of ii) is the observation that by.6) there is a constant a > 0 such that P { gε 0 ) > a} > 0. Define the events A i = gε i) a, We introduce the σ algebras 1 j i 1 cε j ) 1, i ) F 0 = {, Ω} and F i = σ{ε j : i j 0}), i 1. Clearly, A i F i for all i 1. Also, by standard arguments, P {A i F i 1 } = P { gε i ) a} P = P { gε 0 ) a} I 1 j i 1 1 j i 1 cε j ) 1 Fi 1 log cε j ) 0. We claim that I 1 j i 1 log cε j ) 0 = 11 a.s.

12 If E log cε 0 ) > 0, then this follows from the strong law of large numbers, while the Chung Fuchs law cf. Chow and Teicher 1988, p. 148)) applies if E log cε 0 ) = 0. Hence, Corollary 3. of Durrett 1986, p. 40) gives P {A i infinitely often } = 1, and therefore P lim gε i) i 1 j i 1 cε j ) = 0 = 0. This contradicts that the sum in.3) is finite with probability one. Proof of Theorem.. If ν 1, then by Minkowski s inequality we have E X ν ν E i) cε j ) 1/ν ν 1 j i 1 ν = E gε 0 ) ν [E cε 0 ) ν ] i 1)/ν < by assumptions.7) and.8). If 0 < ν < 1, then cf. Hardy, Littlewood and Pólya 1959)) E X ν E gε i) 1 j i 1 cε j ) ν = E gε 0 ) ν [E cε 0 ) ν ] i 1 <, completing the proof of the first part of Theorem.. If ν 1, then by the lower bound in the Minkowski inequality cf. Hardy, Littlewood and Pólya 1959)) and.10) we get E X ν ν E gε i ) cε j ) = E[gε 0 )] ν E[cε 0 )] ν ) i 1. 1 j i 1 Since by.6) and.7) we have 0 < E[gε 0 )] ν <, we get.8). Similarly, if 0 < ν < 1, then E X ν E gε i ) cε j ) ν 1/ν ν 1 j i 1 = E[gε 0 )] ν ν E[cε 0 )] ν ) i 1)/ν and therefore.8) holds. Proof of Theorem.3. The first part is an immediate consequence of Brandt 1986). 1

13 The second part is based on.1) with k = 0. Since by.10) all terms in.1) are nonnegative, we get for all N 1 that 4.) Λσ 0) 1 i N gε i ) 1 j i 1 cε j ). If E log cε 0 ) 0, then the proof of Theorem.1ii) gives that lim sup i gε i ) 1 j i 1 cε j ) a with some a > 0. So by.10) we obtain that contradicting 4.) lim N 1 i N gε i ) cε j ) = 1 j i 1 a.s., 5 Proof of Theorem.4 Since the proof of Theorem.4 is divided into a series of lemmas, we will first outline its structure. The main ideas can be highlighted as follows. First goal of the proof is to show that the partial sums of {y k} can be approximated by the partial sums of a sequence of random variables {ỹk } which are independent at sufficiently large lags. To do so, we establish exponential inequalities for the distance between the volatilities {σ k } and a suitably constructed sequence { σ k }. Consequently, letting ỹ k = σ k ε k, it suffices to find an approximation for the partial sums n k=1 ỹk Eỹ k ). Detailed proofs are given in Lemmas below. Set ξ k = ỹ k Eỹ k. The partial sum n k=1 ξk will be blocked into random variables belonging to two subgroups. The random variables associated with each subgroup form an independent sequence which allows the application of the following simplified version of Theorem 4.4 in Strassen 1965, p. 334). Theorem A. Let {Z i } be a sequence of independent centered random variables and let 0 < κ < 1. If rn) = 1 i N EZ i N ), 1 N< then there is a Wiener process {Wt) : t 0} such that 1 i N rn) κ EZ 4 N <, Z i WrN)) = o rn) κ+1)/4 log rn) ) a.s. N ). 13

14 Lemmas help proving that the assumptions of Theorem A are satisfied, while the actual approximations are given in Lemmas Let 0 < ρ < 1 and define σ k as the solution of Λ σ k) = 1 i k ρ gε k i ) 1 j i 1 cε k j ). First, we obtain an exponential inequality for Λσ k ) Λ σ k ). Lemma 5.1 If 1.1) 1.4),.5),.7) and.8) hold, then there are constants C 1,1, C 1, and C 1,3 such that Proof. Observe that P { Λσ k ) Λ σ k ) exp C 1,1k ρ ) } C 1, exp C 1,3 k ρ ). Λσ k ) Λ σ k ) = k ρ + gε k i ) Using Theorem.7 of Petrov 1995) yields P cε k i ) exp C 1,4 k ρ ) 1 i k ρ 1 j i 1 cε k j ) = 1 i k ρ cε k i )Λσ k k ρ). = P log cε k i ) a) C 1,4 + a)k ρ 1 i k ρ exp C 1,5 k ρ ), where E log cε 0 ) = a < 0 and 0 < C 1,4 < a. Theorem. implies P { Λσ k ) Λ σk ) exp C )} 1,4 kρ { P cε k i ) exp C 1,4 k ρ ) + P Λσk k ρ) exp C1,4 1 i k ρ kρ exp C 1,5 k ρ ) + exp νc ) 1,4 kρ E Λσ0 ) ν C 1,6 exp C 1,7 k ρ ). This completes the proof. )} Lemma 5. If 1.1) 1.4),.5),.7),.8),.11) and.1) hold, then there are constants C,1, C, and C,3 such that P { σ k σ k exp C,1k ρ ) } C, exp C,3 k ρ ). 14

15 Proof. By the mean value theorem, σ k σ k = Λ 1 Λσ k)) Λ 1 Λ σ k)) = where ξ k is between Λσ k) and Λ σ k). By.1), 5.1) 1 Λσ Λ Λ 1 ξ k )) k ) Λ σ k) ), σk σ k C Λσk) γ + Λ σ k) γ Λσ k ) Λ σ k). If γ 0, then Lemma 5. follows from.11) in combination with Lemma 5.1. If γ > 0, then Theorem. and Lemma 5.1 yield { Λσ P k ) γ + Λ σ k ) γ) Λσ k ) Λ σ k ) exp C )} 1,1 kρ P { Λσ k ) Λ σ k) exp C 1,1 k ρ ) } + P C 1, exp C 1,3 k ρ ) + P +P { Λσ k) ν { Λσ k) γ + Λ σ k) γ exp )) 1 ν/γ } exp C1,1 kρ { Λσ k ) + exp C 1,1k ρ ) ) )) ν 1 ν/γ } exp C1,1 kρ C,4 exp C,5 k ρ ), )} C1,1 kρ finishing the proof. Lemma 5.3 If 1.1) 1.4),.5),.7),.8) hold and Λx) = log x, then there are constants C 3,1 and C 3, such that P { σ k σ k exp C 3,1k ρ ) } C 3, k ρν. Proof. By the mean value theorem, σk σ k = elog σ k e log σ k = e ξ k log σ k log σ k ), where ξ k is between log σk and log σ k. Hence, 5.) σ k σ k e log σ k + e log σ k) log σ k log σ k. On the set 1 < log σ k log σ k < 1, we have explog σ k) 3 explog σ k), so by Theorem. and Lemma 5.1, we obtain P { σk σ k exp C 1,1 P kρ )} {3e log σk log σ k log σ k exp C )} 1,1 kρ 15 + C 1, exp C 1,3 k ρ )

16 P { log σ k log σ k exp C 1,1 k ρ ) } { )} +P 3e log C1,1 σ k exp kρ + C 1, exp C 1,3 k ρ ) { C 1, exp C 1,3 k ρ ) + P log σk C } 1,1 kρ log 3 + C 1, exp C 1,3 k ρ ) { ) ν } C 1, exp C 1,3 k ρ ) + P log σk ν C1,1 kρ log 3 + C 1, exp C 1,3 k ρ ) C 3,3 k ρν. This completes the proof. Let ỹ i = σ i ε i, < i <. It is clear that ỹ i and ỹ j are independent if i < j j ρ. The next result shows that it is enough to approximate the sum of the ỹ i. Lemma 5.4 If the conditions of Theorem.4 are satisfied, then max yi ỹi = O1) a.s. 1 k< Proof. Condition.14) implies 1 i k as k. By Lemmas 5. and 5.3 we get 1 i k ε k = o k 1/8+δ)) a.s. σ k σ k = O exp C 4,1k ρ )) a.s. Thus we conclude max 1 k< 1 i k y i and the proof is complete. ỹi 1 i k yi ỹ i a.s. = O1) = We note that under the conditions of Theorem.4 we have σi σ i ε i i /8+δ) exp C 4,1 i ρ ) a.s. = O1), 5.3) P { σ k σ k exp C 4, k ρ ) } C 4,3 k µ, where µ = ρν when Λx) = log x while µ can be chosen arbitrarily when.11) and.1) hold. In both cases µ can be chosen arbitrarily large. 16

17 Lemma 5.5 If the conditions of Theorem.4 are satisfied, then 5.4) and E σ k) 4+δ = O1) with some δ > 0 Eσ k σ k ) = O k µ/), where µ is defined in 5.3). Proof. We first show that E σ k σ k 4+δ = O1). Assume first that.11) and.1) are satisfied. Using 5.1), we get in case of γ > 0 that σ k σ k 4+δ C Λσ k) 4+δ)1+γ) + Λ σ k) 4+δ)1+γ) + Λσ k) 4+δ)γ Λ σ k) 4+δ + Λσ k) 4+δ Λ σ k) 4+δ)γ). It follows from our assumptions that E Λσ k) 4+δ)1+γ) C 5,1. Applying the defintion of σ k, we obtain from the proof of Theorem. Similarly, E Λ σ k) 4+δ)1+γ) E gε k i ) 1 i k ρ E 1 i gε k i ) 1 j i 1 1 j i 1 cε k j ) cε k j ) 4+δ)1+γ) 4+δ)1+γ) C 5,. Λσk ) 4+δ)γ Λ σ k ) 4+δ 4+δ)γ gε k i ) cε k j ) 4+δ gε k i ) cε k j ) 1 i 1 j i 1 1 i k ρ 1 j i 1 4+δ)1+γ) gε k i ) cε k j ) 1 i 1 j i 1 and therefore E Λσk) 4+δ)γ Λ σ k) 4+δ C 5,3. The same argument implies in a similar way E Λσk ) 4+δ Λ σ k ) 4+δ)γ C 5,4. If γ < 0, then 5.1) and.11) yield σk σ k 4+δ C Λσ 5,5 k ) Λ σ k ) 4+δ and similar as before E Λσk) 4+δ C 5,6 as well as E Λ σ k) 4+δ C 5,6. 17

18 If Λx) = log x, then by 5.) we have σ k σ k 4+δ C 5,7 e 4+δ) log σ k + e 4+δ) log σ k) log σ k 4+δ + log σ k 4+δ). So, it is enough to prove that 5.5) Ee η log σ k C5,8 and Ee η log σ k C5,8 with some η > 4 and C 5,8 = C 5,8 η). By assumption.16), Λσk ) gε k i ) i i< 1 j i 1 cε k j ) c i 1 gε k i ). Now Mt) = E expt gε 0 ) ) exists for some t > 4 and therefore E expt Λσk) ) Mtc i 1 ) = exp log Mtc i 1 ). Since 0 < c < 1, we see that tc i 0 as i. For any x > 0, log1 + x) x and Mx) = 1 + Ox) as x 0, so we get the first part of 5.5). A similar argument applies to the second statement. For k 1, define the events A k = { σk σ k exp C 5,9k ρ )} and write σk σ k) = σk σ k) I{A k } + σk σ k) I{A c k}. Clearly, E[σ k σ k ) I{A k }] = O exp C 5,9 k ρ )). Applying 5.3), 5.4) and the Cauchy Schwarz inequality, we arrive at E [ σ k σ k ) I{A c k }] E σ k σ k 4) 1/ PA c k ) 1/ = O k µ/), completing the proof of Lemma 5.5. Lemma 5.6 If the conditions of Theorem.4 are satisfied, then Covy0, y k ) 1 k< is absolutely convergent. 18

19 Proof. Since y 0 and ỹ k are independent for all k > 1, we obtain Ey0 Ey0)y k Eyk) = Ey0 Ey0)y k ỹk + ỹk Eyk) = Ey0 Ey 0 )ỹ k Ey k ) + Ey 0 Ey 0 )y k ỹ k ) = Ey0 y k ỹ k )) Ey 0 Ey k ỹ k ). Since yk ỹ k = σ k σ k )ε k and the random variables σ k σ k, ε k are independent, Lemma 5.5 and Cauchy s inequality give Covy0, yk) = O k µ/4), so Lemma 5.6 is proved on account of µ/4 > 1. Lemma 5.7 If the conditions of Theorem.4 are satisfied, then there is a constant C 7,1 such that for all n, m 1. E n<k n+m ỹk Eỹ k ) C 7,1 m Proof. Set ξ k = y k Ey k and ξ k = ỹ k Eỹ k. It is easy to see that E n<k n+m ξ k = n<k,l n+m Eξ k ξ l + Lemma 5.5 and Cauchy s inequality imply n<k,l n+m Similarly, E ξ k ξ l ξ l ) = = n<k,l n+m n<k,l n+m = O1) n k,l n+m n<k,l n+m E ξ k ξ l ξ l ) + E ξ k E ξ l ξ l ) ) 1/ n<k,l n+m E σ 4 k Eε 4 k) 1/ Eε 4 l Eσ l σ l )) 1/ n<k,l n+m l µ/4 = O1)m E ξ l ξ k ξ k ) = O1)m. By the stationarity of {ξ k : < k < }, we have n k,l n+m Eξ k ξ l = n k,l n+m n<l< Eξ 0 ξ k l = O1)m Eξ l ξ k ξ k ). l µ/4 = O1)m. by an application of Lemma

20 Lemma 5.8 If the conditions of Theorem.4 are satisfied, then there are constants n 0, m 0 and C 8,1 > 0 such that for all n n 0 and m m 0. E n<k n+m ỹk Eỹk) Proof. Following the proof of Lemma 5.7, we get Since 1 m E n<k n+m n<k,l n+meξ k ξ l = 1 m ξ k n<k,l n+m n<k,l n+m C 8,1 m Eξ k ξ l C 8, m Eξ 0 ξ k l Var y 0 + n<l< 1 k< l µ/4. Covy0, y k ) = σ > 0, the proof is complete. Lemma 5.9 If the conditions of Theorem.4 are satisfied, then there is a constant C 9,1 such that E 4 5.6) ỹk Eỹk) C 9,1 m for all n, m 1. n<k n+m Proof. We prove 5.6) using induction. If m = 1, then by 5.4) we have Eỹ k Eỹ k) 4 C 9,, for all k 1. We assume that 5.6) holds for m and for all n. We have to show that 5.7) E n<k n+m+1 4 ξ k C 9,1 m + 1) for all n. Towards this end, let denote integer part, m ± ρ) = m/ ± m ρ and set S 1 n, m) = n+ m ρ) k=n+1 ξ k, S n, m) = By the triangle inequality, we have E 1 j 3 S j n, m) 4 1/4 n+m+1 k=n+ m + ρ) +1 ξ k, S 3 n, m) = n+ m + ρ) k=n+ m ρ) E [S 1 n, m) + S n, m)] 4) 1/4 + E [S3 n, m)] 4) 1/4. 0 ξ k.

21 Also, by 5.4), E [S3 n, m)] 4) 1/4 C9,3 m ρ. Let A k = { σ k σ k exp C 9,4k ρ )} and write ξ k = ξ k + ξ k ξ k )I{A k }+ ξ k ξ k )I{A c k }. Let U = {n + 1, n +,..., n + m ρ) } {n + m + ρ),...,n + m + 1}. Then, E [S1 n, m) + S n, m)] 4) 1/4 E ξ k k U 4 1/4 + E k U ξ k ξ k )I{A k } It follows from the definition of the A k that 4 1/4 4 E k U ξ k ξ k )I{A k } + E C 9,5. ξ k ξ k )I{A c k } k U Moreover, by 5.4), 5.3) and Minowski s and Hölder s inequalities we obtain that E 4 1/4 ξ k ξ k )I{A c k } E ξ k ξ k ) 4 I{A c k }) 1/4 k U k U k U E ξ k ξ k ) 4+δ ) 1/4+δ) P {A c k}) δ/4+δ) C 9,6 Observe that by the stationarity of the ξ k, E 4 ξ k = E k U Arguing as before, we estimate E ξ k + 1 k m ρ) Since the partial sums ξ k 1 k m ρ) m + ρ) k m+1 and 4 1/4 ξ k 1 k m ρ) m + ρ) k m+1 with some m 0 and E ξ k = 0, we obtain that 4 E ξ k + ξ k 1 k m ρ) = E 1 k m ρ) m + ρ) k m+1 4 ξ k + E m + ρ) k m+1 E ξ k + 1 k< 4 ξ k m + ρ) k m+1 1 k m + ρ) ξ k + k µδ/4+δ) C 9,7.. m + ρ) k m+1 4 1/4 ξ k ξ k are independent, if m m 0 4 ξ k + 6E 1 1 k m ρ) ξ k E m + ρ) k m+1 4 1/4. +C 9,8. ξ k.

22 Using the induction assumption we get 4 m E ξ k C 9,1 4 1 k m ρ) Lemma 5.7 yields 6E 1 k m ρ) ξ k E and E m + ρ) k m+1 Since we can assume that C 9,1 6C 7,1, we conclude E thus completing the proof. 1 k m ρ) ξ k + m + ρ) k m+1 m + ρ) k m+1 ξ k 6C7,1 m 4. 4 ξ k 3 4 C 9,1m, 4 m ξ k C 9,1 4. Lemmas contain upper and lower bounds for the second and fourth moments of the increments of the partial sums of the ỹi Eỹ i. As we will see, these inequalities together with the independence of the ỹ j and ỹ i, where i < j j ρ, are sufficient to establish the strong approximation in Theorem.4. According to Lemma 5.4 it suffices to approximate ỹi Eỹi ). Let η > 1 + ρ, ηρ < τ < η 1 and set X k = k η i k+1) η k+) τ ξi and Y k = k+1) η k+) τ <i<k+1) η ξi, where ξ i = ỹ i Eỹ i. Then {X k} and {Y k } are sequences each consisting of independent random variables. To show that X k and X k+1 are independent, consider the closest lag, i.e. the index difference of the first ξ i in X k+1 and of the last ξ i in X k. We obtain that k + 1) η [k + 1) η k + ) τ ] > k + 1) τ > k + 1) ηρ which implies the assertion. Similar arguments apply to {Y k }. Finally, let 5.8) rn) = E X k 1 k N and r N) = E Y k 1 k N Lemma 5.10 If the conditions of Theorem.4 are satisfied, then there is a Wiener process {W 1 t) : t 0} such that 1 k N for any η 1)/η) < κ < 1. X k W 1 rn)) = o rn) κ+1)/4 log rn) ) a.s.

23 Proof. It follows from the definition of ξ i that X 1, X,... are independent random variables with mean 0. By Lemmas 5.7 and 5.8, there are C 10,1 and C 10, such that EX k C 7,1 k + 1) η k + ) τ k η ) C 10,1 k η 1 and 5.9) EX k C 8,1 k + 1) η k + ) τ k η ) C 10, k η 1. Using Lemma 5.9, we can find C 10,3 such that 5.10) EX 4 k C 9,1 k + 1) η k + ) τ k η ) C 10,3 k η 1). Now, 5.9) and 5.10) imply κ EXi 1 k< 1 i k EXk 4 C 10,4 k η 1) κη < 1 k< and the result follows from Theorem A. Lemma 5.11 If the conditions of Theorem.4 are satisfied, then there is a Wiener process {W t) : t 0} such that 1 k N for any τ + 1)/1 + τ)) < κ < 1. Y k W r N)) = o r N) κ +1)/4 log r N) ) Proof. Following the proof of Lemma 5.10, we have EYk C 11,1 k τ and EYk 4 C 11, k τ. Hence, EY κ i EYk 4 C 11,3 k τ τ+1)κ <, 1 k< 1 i k so the assertion follows from Theorem A. 1 k< The next lemma gives an upper bound for the increments of i nỹ i Eỹ i ). a.s. Lemma 5.1 If the conditions of Theorem.4 are satisfied, then max k η j k+1) η ξi = O rk) κ+1)/4 log rk) ) j i k+1) η a.s., where κ is specified in Lemma

24 Proof. On account of Lemma 5.9 we can apply Theorem 1. of Billingsley 1968, p. 94), resulting in P So, max ξi k η j k+1) η j i k+1) η P max ξi k η j k+1) η j i k+1) η Since η 1)/η < η 1)/η) < κ, we obtain that 1 k< P λ C 1,1 k + 1) η k η ) C 1, λ 4 λ 4 kη 1). kηκ+1)/4 log k C 1,k η 1) ηκ+1). max ξi k η j k+1) η j i k+1) η kηκ+1)/4 log k < and therefore using the Borel Cantelli lemma the proof is complete. For any real n 1 not necessarily an integer), we introduce the notations T n = E ξ k 1 k n and Tn = E 1 k n ξ k. The final lemma contains certain properties of the quantities T n and T n. Lemma 5.13 Let N k = k + 1) η. Then we have, with some constant C > 0, i) T Nk rk) Crk) 1/ r k) 1/ for N k < n N k+1, ii) T n T Nk Ck η 1)/ for N k < n N k+1, iii) T n T n C n for n 1. 1/ Proof. By Minkowski s inequality, we have T N k rk) 1/ r k) 1/ and thus T Nk rk) r k) 1/ 1/ T N k + rk) 1/ ) r k) 1/ rk) 1/ + r k) 1/ ), proving i). The proof of ii) is similar, using Lemma 5.7. Finally, to prove iii) we apply Minkowski s inequality again to get T 1/ n 1/ T n n E[ξk ξ k ] ) 1/ k=1 and, using the independence of ε k and σ k σ k and Lemma 5.5, we obtain E[ξk ξ k ] ) 1/ E[σ k σ k ] E[ε 4 k ]) 1/ = O k µ/4 ). 4

25 As µ can be chosen arbitrarily large, we get T 1/ T n = On) by Lemma 5.7. n T 1/ n Finally, we are in a position to give the proof of Theorem.4. = O1), which implies iii), since Proof of Theorem.4. Putting together Lemmas 5.10, 5.11 and the law of the iterated logarithm for W yields 5.11) ξi W 1 rn)) = O rn) κ+1)/4 log rn) ) a.s. 1 i N+1) η provided the constants involved are chosen according to the side conditions 5.1) ηκ + 1) > τ + 1) and η 1 η < κ < 1, which assure that the sum of the Y k is of smaller order of magnitude than the remainder term in Lemma Observe that rn) N η and r N) N τ+1, where means that the ratio of the two sides is between positive constants. Letting now S t = ξ ξ t, t 0, we get for N k 1 < n N k, using Lemmas 5.1, 5.13, relation 5.11) and standard estimates for the increments of Wiener processes see, e.g., Theorem 1..1 of Csörgő and Révész 1981,p. 30)), S n W 1 T n ) S n S N k + S N k W 1 rk)) + W 1 rk)) W 1 T Nk ) + W 1 T Nk ) W 1 T n ) + W 1 T n ) W 1 T n ) = O rk) κ+1)/4 log rk) ) + O rk) κ+1)/4 log rk) ) +O rk) 1/4 r k) 1/4 log k ) + O k η 1)/4 log k ) + O n 1/4 log n ) 5.13)= O n κ+1)/4 log n ) + O n η+τ+1)/4η log n ) + O n η 1)/4η log n ) + O n 1/4 log n ). In the last step we expressed the remainder terms with n, using the fact that k n 1/η. Choose now η = 3/, τ near 0 and κ very near to, but larger than, /3. In this case the relations 5.1) are satisfied and hence 5.13) yields 5.14) S n W 1T n ) = O n 5/1+ε) for any ε > 0. Now {ξ k } is a stationary sequence with zero mean and covariances a.s. Eξ 0 ξ k ) = Ok µ ) for any µ > 0, from which easily follows that T n = E = n σ + O1), ξ k 1 k n where σ is defined in.13). Thus, on applying Theorem 1..1 of Csörgő and Révész 1981,p. 30), we arrive at W 1 T n ) = W 1 n σ ) + Olog n) a.s., so Theorem.4 is readily proved on setting Wt) = σ 1 W 1 σ t). 5

26 References [1] Aue, A., and Horváth, L. 004). Delay time in sequential detection of change. Statistics and Probability Letters 67, [] Aue, A., and Horváth, L. 005). A note on the existence of a stationary augmented GARCH process. Preprint, University of Utah. [3] Aue, A., Horváth, L., Hušková, M., and Kokoszka, P. 005). Change Point Monitoring in Linear Models with Conditionally Heteroskedastic Errors. Preprint, University of Utah. [4] Berkes, I., Gombay, E., Horváth, L., and Kokoszka, P. 004). Sequential change point detection in GARCHp, q) models. Econometric Theory 0, [5] Berkes, I., and Horváth, L. 001). Strong approximation for the empirical process of a GARCH sequence. The Annals of Applied Probability 11, [6] Berkes, I., Horváth, L., Kokoszka, P., and Shao, Q. M. 005). On the almost sure convergence of the Bartlett estimator. Periodica Mathematica Hungarica to appear). [7] Billingsley, P. 1968). Convergence of Probability Measures. Wiley, New York. [8] Bollerslev, T. 1986). Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics 31, [9] Bougerol, P., and Picard, N. 199). Stationarity of GARCH processes and of some nonnegative time series. Journal of Econometrics 5, [10] Brandt, A. 1986). The stochastic equation Y n+1 = A n Y n + B n with stationary coefficients. Advances in Applied Probability 18, [11] Carrasco, M., and Chen, X. 00). Mixing and moment properties of various GARCH and stochastic volatility models. Econometric Theory 18, [1] Chow, Y.S., and Teicher, H. 1988). Probability Theory nd ed.). Springer, New York. [13] Csörgő, M., and Horváth, L. 1997). Limit Theorems in Change Point Analysis. Wiley, Chichester. 6

27 [14] Csörgő, M., and Révész, P. 1981). Strong Approximations in Probability and Statistics. Academic Press, New York. [15] Darling, D.A., and Erdős, P. 1956). A limit theorem for the maximum of normalized sums of independent random variables. Duke Mathematical Journal 3, [16] Ding, Z., Granger, C.W.J., and Engle, R. 1993). A long memory property of stock market returns and a new model. Journal of Empirical Finance 1, [17] Duan, J. C. 1997). Augmented GARCHp,q) process and its diffusion limit. Journal of Econometrics 79, [18] Durrett, R. 1991). Probability: Theory and Examples. Wadsworth & Brooks/Cole, Pacific Grove, CA. [19] Eberlein, E. 1986). On strong invariance principles under dependence assumptions. The Annals of Probability 14, [0] Engle, R.F. 198). Autoregressive conditional heteroskedasticity with estimates of the variance of U.K. inflation. Econometrica 50, [1] Engle, R., and Ng, V. 1993). Measuring and testing impact of news on volatility. Journal of Finance 48, [] Geweke, J. 1986). Modeling persistence of conditional variances: a comment. Econometric Reviews 5, [3] Giraitis, L., Kokoszka, P., Leipus, R., and Teyssiére, G. 003). Rescaled variance and related tests for long memory in volatility and levels. Journal of Econometrics 11, [4] Glosten, L., Jagannathan, R., and Dunke, D. 1993). Relationship between expected value and the volatility of the nominal excess return on stocks. Journal of Finance 48, [5] Hardy, G.H., Littlewood, J.E., and Pólya, G. 1959). Inequalities nd ed.). Cambridge University Press, Cambridge, U.K. [6] Horváth, L., Kokoszka, P., and Zhang, A. 005). Monitoring constancy of variance in conditionally heteroskedastic time series. Econometric Theory, to appear. 7

28 [7] Horváth, L., and Steinebach, J. 000). Testing for changes in the mean or variance of a stochastic process under weak invariance. Journal of Statistical Planning and Inference 91, [8] Kuelbs, J., and Philipp, W. 1980). Almost sure invariance principles for partial sums of mixing B valued random variables. The Annals of Probability 8, [9] Lo, A. 1991). Long run memory in stock market prices. Econometrica 59, [30] Nelson, D.B. 1990). Stationarity and persistence in the GARCH1,1) model. Econometric Theory 6, [31] Nelson, D.B. 1991). Conditional heteroskedasticity in asset returns: a new approach. Econometrica 59, [3] Petrov, V.V. 1995). Limit Theorems of Probability Theory. Clarendon Press, Oxford, U.K. [33] Révész, P. 1990). Random walk in random and nonrandom environments. World Scientific Publishing Co., Teaneck, NJ. [34] Schwert, G.W. 1989). Why does stock market volatility change over time? Journal of Finance 44, [35] Strassen, V. 1965). Almost sure behaviour of sums of independent random variables and martingales. In: Proceedings of the Fifth Berkeley Symposium in Mathematical Statistics and Probability, [36] Straumann, D., and Mikosch, T. 005). Quasi maximum likelihood estimation in heteroscedastic time series: a stochastic recurrence equations approach. The Annals of Statistics, to appear. [37] Taylor, S. 1986). Modelling Financial Time Series. Wiley, New York. [38] Zakoian, J. M. 1994). Threshold heteroskedastic models. Journal of Economic Dynamics and Control 18,