Coditioal-Sum-of-Squares Estimatio of Models for Statioary Time Series with Log Memory.M. Robiso Lodo School of Ecoomics The Sutory Cetre Sutory ad Toyota Iteratioal Cetres for Ecoomics ad Related Disciplies Lodo School of Ecoomics ad olitical Sciece Discussio paper Houghto Street No. EM/2006/505 Lodo WC2A 2AE September 2006 Tel: 020 7955 6679 Research supported by ESRC Grat R000239936.
Abstract Employig recet results of Robiso (2005) we cosider the asymptotic properties of coditioal-sum-of-squares (CSS) estimates of parametric models for statioary time series with log memory. CSS estimatio has bee cosidered as a rival to Gaussia maximum likelihood ad Whittle estimatio of time series models. The latter kids of estimate have bee rigorously show to be asymptotically ormally distributed i case of log memory. However, CSS estimates, which should have the same asymptotic distributioal properties uder similar coditios, have ot received comparable treatmet: the trucatio of the ifiite autoregressive represetatio iheret i CSS estimatio has bee essetially igored i proofs of asymptotic ormality. Ulike i short memory models it is ot straightforward to show the trucatio has egligible effect. AMS 2000 subject classificatio: 62M0. Keywords ad phrases: Log memory, coditioal-sum-of-squares estimatio, cetral limit theorem, almost sure covergece. The author. All rights reserved. Short sectios of text, ot to exceed two paragraphs, may be quoted without explicit permissio provided that full credit, icludig otice, is give to the source.
Itroductio Cosider a real-valued, strictly ad covariace statioary time series x t, t 2 Z. It is believed that x t has a parametric autoregressive (AR) represetatio j ( 0 )x t j = " t ; t 2 Z: (.) j=0 Here " t is a sequece of zero-mea, ucorrelated ad homoscedastic radom variables, with variace 2 0, the j () are give fuctios with p vector argumet, 0 is a ukow p vector, ad 0 () for all. The rage of structures f j ()g covered by (.) is very broad, but of iterest to us are oes which allow x t to have log memory. Usually, these are "fractioal", where it is assumed that the fuctio (s; ) = j ()s j ; (.2) j=0 with complex-valued argumet s o the uit circle, is of form where () is a scalar fuctio of such that (s; ) = ( s) () (s; ); (.3) 0 < ( 0 ) < 2 (.4) ad It follows that x t has spectral desity of form f() = 0 < j (s; 0 )j < ; jsj = : (.5) 2 0 j (e i ; 0 )j 2 = e i 2(0) 2 0 j (e i ; 0 )j 2 : (.6) The leadig choice of is a ratioal fuctio of s, i which case x t is said to be a fractioal autoregressive itegrated movig average (FARIMA) model; ( 0 ) is caled the memory parameter. Leadig methods of estimatio of 0, give observatios x ; :::; x, are Gaussia maximum likelihood (ML), ad approximatios thereto. They are "approximatios" i the sese that uder similar coditios they have the same asymptotic ormal distributio as ML, ad are thus asymptotically e ciet uder Gaussiaity. At the same time, uder may departures from Gaussiaity, though the e ciecy is lost the limit ormal distributio of all these estimates is ua ected. Assumig Gaussiaity, asymptotic ormality of oe form of approximatio, a Whittle estimate ivolvig itegratio over frequecy, was rst established by Fox ad Taqqu (986), ad the by Dahlhaus (989) i case of ML estimatio. Giraitis ad Surgailis (990) established asymptotic ormality for the estimate 2
cosidered by Fox ad Taqqu (986) whe " t eed ot be Gaussia but is idepedet ad idetically distributed with ite fourth momet. Due to the pole i the spectral desity at = 0 (see (.6)), the asymptotic ormality proofs are cosiderably more challegig tha those of Haa (973) for short memory time series models, icisive though these were for such models. A alterative estimate that has bee cosidered i the literature is coditioalsum-of-squares (CSS) estimatio, which was previously employed by Box ad Jekis (970) for short memory time series models. De e ad estimate 0 by e t () = t j=0 s () = j ()x t j ; (.7) e 2 t (); (.8) ^ = arg mi 2 s (); (.9) where R p is a compact set. Oe ca motivate ^ by the hope that s ( 0 ) is a good approximatio to "2 t, which is itself proportioal to the expoet i the desity fuctio of idepedet idetically distributed zero-mea ormal variates. Thus oe hopes that (after ceterig at 0 ad 2 ormig) ^ has the same limit distributioal properties as the Gaussia ML ad Whittle estimates metioed previously. Give a iitial cosistecy proof of ^, a stadard approach to provig asymptotic ormality etails applyig the mea value theorem to r (^ ) about 0, where r () = @s () = 2 h t ()e t (); (.0) @ for h t () = @e t() : (.) @ The mai part of the proof the ivolves establishig that 2 r ( 0 ) coverges i distributio to a zero-mea ormal vector. If the " t are assumed to be coditioally homoscedastic martigale di ereces, ad coditios esurig that h t () has ite variace are imposed, such covergece is easily see to hold (see e.g. Brow (97)) for r ( 0 ) = 2 h t " t ; (.2) where h t = h t ( 0 ). However this is oly useful if also r( 0 ) r ( 0 ) = o p 2 ; (.3) 3
i other words, if the e ect of replacig e t = e t ( 0 ) by " t is su cietly small. Ulike the h t " t, the h t e t ad h t (e t " t ) are ot zero-mea, orthogoal radom variables. We ca employ the Schwarz iequality: E jr ( 0 ) r ( 0 )j 2 h E(e t " t ) 2 E kh t ( t ( 0 )k 2i 2 : (.4) The if, say, it were true that E(e t " t ) 2 = O p (t ) for some > 0, the right had side of (.4) would be O p 2 2, ad (.3) established. For short memory models E(e t " t ) 2 typically decays fast eough, ideed eve expoetially. But uder quite geeral coditios permittig log memory (see Robiso (2005)), E(e t " t ) 2 Kt (.5) oly, where K is a arbitrarily large geeric costat, which is isu ciet to establish (.3) usig (.4). A more delicate proof of (.3) is required, ad this was give by Robiso (2005). As discussed there, this delicacy relates to that see i the proofs of Fox ad Taqqu (986) ad others for alterative estimates of 0. Ideed, give that these estimates ad CSS should have the same limit distributioal properties, it would be extraordiary if the proof for CSS were very much easier tha for the other estimates. A cetral limit theorem for ^ is give i Sectio 3. rior to that, i the followig sectio, we provide the almost covergece of ^ (uder somewhat more geeral coditios). Haa (973) proved this for various estimates, assumig strict statioarity ad ergodicity, which is cosistet with log memory. However, he did ot cover CSS estimatio. 2 Almost Sure Covergece I the preset sectio we do ot require that x t ecessarily has spectral desity of form (.6), with (.5) holdig, but simply that it is a zero-mea strictly statioary, ergodic process with AR represetatio (.), with the setece after (.) holdig, ad also 0 2 ; for all 2 f 0 g (s; ) 6= (s; 0 ) (2.) o a subset of jsj = of positive measure, j(s; )j is cotiuous i for all s : jsj = ; ad sup j j ()j < : (2.2) j=0 2 Coditio (2.) is a stadard ideti ability coditio, ad (2.2) is reasoable i that log memory models (e.g. (.6), such as FARIMAs) typically have AR represetatios with summable coe ciets. Note that this setup allows the 4
spectral desity to have poles at o-zero frequecies (as i certai cyclic ad seasoal models), whereas (.6) does ot, i view of (.5). Theorem Uder the above coditios lim ^ = 0 ; a:s: (2.3)! roof. Theorem of Haa (973) ad Theorem of Fox ad Taqqu (986) cover the estimate ~ = arg mi sy (); (2.4) where s y () is the objective fuctio for the itegral form of Whittle estimate, i.e. 2 N () of Haa (973) or 2 N () of Fox ad Taqqu (986). We ca write where s y () = c (0) 0 () + 2 c (j) j (); (2.5) c (j) = j x t x t+j ; 0 j ; (2.6) j () = k () k+j (): (2.7) k=0 From Theorem of Haa (973), ad its proof, it is clear that it su ces to show that lim s y () s () = 0; a:s: (2.8) Now sup! s y () s () = + 2 x 2 t k= t+ j x t x t+j 2 k() k= t j+ k () k+j () = 4 a i (); (2.9) i= where ( a () = (0) j 2 j() + a 2 () = x 2 t (0) a 3 () = 2 a 4 () = 2 (j) j ( j= k= t+ k= t j+ 2 j() j (x t x t+j (j)) ) ; (2.0) 2 k(); (2.) k () k+j (); (2.2) k= t j+ k () k+j () ) ;(2.3) 5
where It remais to prove (j) = cov(x 0 ; x j ): (2.4) lim sup ja i ()j = 0; i = ; 2; 3; 4: (2.5)! 2 As the proofs for i = ; 2 are similar to but simpler tha those for i = 3; 4, we give oly the latter. We have ( 2 sup ja 3 ()j 2 j(j)j sup j j ()j) : (2.6) j=0 2 The quatity i braces is ite ad sice, by the Riema-Lebesgue theorem, existece of a spectral desity implies lim j! (j) = 0, it follows from the Toeplitz lemma that (2.6)! 0 as!. Next, by summatio-by-parts a 4 () = 2 +2 j t fc t(j) (j)g t j+ () t+ () j fx t x t+j (j)g k () k+j (): (2.7) The modulus of the rst term o the right has supremum, over, bouded by k= K sup j jc t (j) (j)j sup j t+ ()j (2.8) usig (2.2). Usig (2.2) agai, ad Theorem of Haa (974) ad the Toeplitz lemma, it follows that (2.8) is o() a.s. The secod term i (2.7) ca be similarly hadled. 3 Asymptotic Normality We assume ow i additio that x t has spectral desity (.6), with (.4), (.5) satis ed, that 0 is a iterior poit of, that the " t i (.) are idepedet with zero mea, variace 2 0 ad uiformly bouded fourth momet, that (s; ) is twice cotiuously di eretiable i, ad that the matrix = R 2 is positive de ite. log e i 2 2 @ @ log e i ; 0 log e i 2 2 @ @ log e i ; 0 0 d (3.) 6
Theorem 2 Uder the above coditios, as! 2 (^ 0 ) coverges i distributio to a p-variate ormal vector with zero mea ad covariace matrix. roof. As discussed i Sectio, we have 0 = r (^ ) = r ( 0 ) + ~ T (^ 0 ); (3.2) where T ~ is the matrix formed by evaluatig, for i = ; :::; p, the i-th row of the matrix T () = @ 2 =@@ 0 s () at = ~ i, where ~ i 0 ^ 0 ; k:k deotig Euclidea orm. De e j = @ @ j(; ); (3.3) so that ad de e also Write r ( 0 ) h t = t j x t j ; (3.4) t = j x t j ; (3.5) r = t " t : (3.6) r = r + r 2 + r 3, where r = 2 (h t t )" t ; (3.7) r 2 = 2 t (e t " t ); (3.8) r 3 = 2 (h t t )(e t " t ): (3.9) We show that r i = o p ( 2 ), i = ; 2; 3. To deal with r, we may write h t t = j x t j = jt " j ; (3.0) j=t where Sice jt = E kh t t k 2 = 2 0 j k=0 k+j j k : (3.) jt 2 (log t) 2 K t (3.2) 7
as oted o p.824 of Robiso (2005), ad " t is idepedet of h t t, it follows that E kr k 2 K 2 t K log 2 : (3.3) Next, we ca write where e t " t = jt " j ; (3.4) jt = j k=0 Thus, from Lemma 6 of Robiso (2005), Fially, E kr 3 k k+j t k : (3.5) E kr 2 k 2 (log )3 K 2 : (3.6) K K E kh t t k 2 E (e t " t ) 2 2 log t t (log )2 ; (3.7) usig (3.2) ad also Lemma 4 of Robiso (2005). This completes the proof that r i = o p ( 2 ), i = ; 2; 3. The remaider of the proof is easier, ad more stadard, ad is omitted. ACKNOWLEDGEMENT I thak a referee for a careful readig of the paper. REFERENCES BOX, G.E.. ad JENKINS, G.M. (970). Time Series Aalysis, Forecastig ad Cotrol. Holde-Day, Sa Fracisco. BROWN, B.M. (97). Martigale cetral limit theorems. A. Math. Statist. 42, 59-66. DAHLHAUS, R. (989). E ciet parameter estimatio for self-similar processes. A. Statist. 7 749-766. (Correctio A. Statist., forthcomig.) FOX, R. ad TAQQU, M.S. (986). Large-sample properties of parameter estimates for strogly depedet statioary Gaussia time series. A. Statist. 4 57-532. GIRAITIS, L. ad SURGAILIS, D. (990). A cetral limit theorem for quadratic forms i strogly depedet liear variables ad its applicatio to asymptotic ormality of Whittle s estimate. robab. Theory Related Fields 86 87-04. 8
HANNAN, E.J. (973). The asymptotic theory of liear time series models. J. Appl. robability 0 30-45. (Correctio: J. Appl. robability 0, 93.) HANNAN, E.J. (974). The uiform covergece of autocovariaces. A. Statist. 2 803-806. ROBINSON,.M. (2005). E ciecy improvemets i iferece o statioary ad ostatioary fractioal time series. A. Statist. 33, 800-842. 9