Iferece for Alteratig Time Series Ursula U. Müller 1, Ato Schick 2, ad Wolfgag Wefelmeyer 3 1 Departmet of Statistics Texas A&M Uiversity College Statio, TX 77843-3143, USA (e-mail: uschi@stat.tamu.edu) 2 Departmet of Mathematical Scieces Bighamto Uiversity Bighamto, NY 13902-6000, USA (e-mail: ato@math.bighamto.edu) 3 Mathematisches Istitut Uiversität zu Köl Weyertal 86-90 50931 Köl, Germay (e-mail: wefelm@math.ui-koel.de) Abstract. Suppose we observe a time series that alterates betwee differet autoregressive processes. We give coditios uder which it has a statioary versio, derive a characterizatio of efficiet estimators for differetiable fuctioals of the model, ad use it to costruct efficiet estimators for the autoregressio parameters ad the iovatio distributios. We also study the cases of equal autoregressio parameters ad of equal iovatio desities. Keywords: Autoregressio, Local asymptotic ormality, Semiparametric model, Efficiecy, Adaptivity. 1 Itroductio By a alteratig AR(1) process of period m we mea a time series X t, t = 0, 1,..., that alterates periodically betwee m possibly differet AR(1) processes, X jm+k = ϑ k X jm+k 1 + ε jm+k, j = 0, 1,..., k = 1,..., m, (1) where the iovatios ε t, t N, are idepedet with mea zero ad fiite variaces, ad ε jm+k has a positive desity f k. The the m-dimesioal process X j = (X (j 1)m+1,..., X jm ), j N, is a homogeeous Markov chai. Its trasitio desity from X j 1 to X j = x = (x 1,..., x m ) depeds oly o the last compoet of X j 1, say x 0, ad is give by m (x 0, x) f k (x k ϑ k x k 1 ). Note that a alteratig AR(1) process is ot a multivariate autoregressive process, which would require a represetatio X j = ΘX j 1 + ε j for a matrix Θ ad i.i.d. vectors ε j.
2 Müller et al. If we replace X jm+k 1 i (1) by its autoregressive represetatio ad iterate this m 1 times, we arrive at the represetatio where X jm+k = τ(ϑ)x (j 1)m+k + η jm+k, j N, (2) ϑ = (ϑ 1,..., ϑ m ), τ(ϑ) = m ϑ k, η jm+k = ad, settig ϑ s = ϑ t if s = t mod m, t 1 τ kt (ϑ) = ϑ k s. s=0 m 1 t=0 τ kt (ϑ)ε jm+k t, I particular, τ k0 (ϑ) = 1 ad τ k,jm (ϑ) = τ j (ϑ). The iovatios η jm+k, j N, i (2) are idepedet with positive desity. Hece for each k = 1,..., m the subseries X jm+k, j = 0, 1,..., is AR(1) ad a irreducible ad aperiodic Markov chai, ad positive Harris recurret if ad oly if τ(ϑ) < 1. I particular, we do ot eed ϑ k < 1 for all k. We obtai that the m-dimesioal Markov chai X j, j N, is irreducible ad aperiodic, ad positive Harris recurret if ad oly if τ(ϑ) < 1. I this case we also have ifiite-order movig average represetatios X jm+k = τ kt (ϑ)ε jm+k t, j = 0, 1,..., k = 1,..., m. (3) t=0 I the followig sectios we derive efficiet estimators for submodels of alteratig AR(1) processes. We treat depedecies betwee the autoregressio parameters ad also cosider the cases of equal autoregressio parameters ad of equal iovatio desities. I Sectio 2 we give coditios uder which the alteratig AR(1) model is locally asymptotically ormal, ad characterize efficiet estimators of vector-valued fuctioals. I Sectio 3 we costruct efficiet estimators for the autoregressio parameters ad the iovatio distributios. Sectio 4 cosiders submodels with equal iovatio desities. 2 Characterizatio of efficiet estimators I order to describe possible depedecies betwee the autoregressio parameters ϑ 1,..., ϑ m, we reparametrize them as follows. Let p m ad A R p ope, let ϑ : A R m, ad set ϑ k = ϑ k (ϱ) for ϱ A. Set f = (f 1,..., f m ). Our model is semiparametric; its distributio is determied by (ϱ, f). Fix ϱ A with τ(ϑ(ϱ)) < 1. Assume that ϑ : A R m has cotiuous partial derivatives at ϱ, ad write ϑ for the m p matrix of partial
Iferece for Alteratig Time Series 3 derivatives ad ϑ k for its k-th row. Assume that ϑ is of full rak. Fix iovatio desities f 1,..., f m. Assume that the f k are absolutely cotiuous with a.e. derivative f k ad fiite Fisher iformatio J k = E[l 2 k (ε k)], where l k = f k /f k. Itroduce perturbatios ϱ t = ϱ + 1/2 t with t R p, ad f kuk (x) = f(x)(1 + 1/2 u k (x)) with u k i the space U k of bouded measurable fuctios such that E[u k (ε k )] = 0 ad E[ε k u k (ε k )] = 0. These two coditios guaratee that f kuk is a mea zero probability desity for sufficietly large. The trasitio desity from X jm+k 1 = x k 1 to X jm+k = x k is f k (x k ϑ k x k 1 ). The perturbed trasitio desity (x k 1, x k ) f kuk (x k ϑ k (ϱ t )x k 1 ) is Helliger differetiable with derivative (x k 1, x k ) ϑ k tx k 1 l k (x k ϑ k x k 1 ) + u k (x k ϑ k x k 1 ). Here ad i the followig we write ϑ for ϑ(ϱ). Set U = U 1 U m, u = (u 1,..., u m ) ad f u = (f 1u1,..., f mum ). Suppose we observe X 0, X 1,..., X. Let P ad P tu deote their joit laws uder (ϱ, f) ad (ϱ t, f u ), respectively. Followig [Koul ad Schick, 1997], who treat oalteratig autoregressio, we obtai local asymptotic ormality log dp tu dp = 1/2 1 2 t ϑ D ϑt 1 2 ( ϑk tx jm+k 1 l k (ε jm+k ) + u k (ε jm+k ) ) E[u 2 k(ε k )] + o p (1), (4) where D is the diagoal matrix with etries E[X1 2 ]J 1,..., E[Xm]J 2 m. Here we have used that X 0 l 1 (ε 1 ),..., X m 1 l m (ε m ), u 1 (ε 1 ),..., u m (ε m ) are ucorrelated. We ca ow characterize efficiet estimators as follows, usig results origially due to Hájek ad LeCam, for which we refer to Sectio 3.3 of the moograph [Bickel et al., 1998]. Let Ūk deote the closure of U k i L 2 (f k ) ad set Ū = Ū1 Ūm. The squared orm of (t, u) o the right-had side of (4) determies how difficult it is, asymptotically, to distiguish betwee (ϱ, f) ad (ϱ t, f u ). It defies a ier product o R p Ū. A realvalued fuctioal ϕ of (ϱ, f) is called differetiable at (ϱ, f) with gradiet (t ϕ, u ϕ ) R p Ū if 1/2 (ϕ(ϱ t, f u ) ϕ(ϱ, f)) t ϕ ϑ D ϑt + E[u ϕk (ε k )u k (ε k )] (5) for all (t, u) R p U. A estimator ˆϕ of ϕ is called regular at (ϱ, f) with limit L if 1/2 ( ˆϕ ϕ(ϱ t, f u )) L uder P tu, (t, u) R p U. (6)
4 Müller et al. The covolutio theorem of Hájek ad LeCam says that L is distributed as the covolutio of some radom variable with a ormal radom variable N that has mea 0 ad variace t ϑ ϕ D ϑt ϕ + E[u 2 ϕk(ε k )]. This justifies callig ˆϕ efficiet if L is distributed as N. A estimator ˆϕ of ϕ is called asymptotically liear at (ϱ, f) with ifluece fuctio g if g L 2 (P 1 ) with E(g(X 0, X 1 ) X 0 ) = 0 ad 1/2 ( ˆϕ ϕ(ϱ, f)) = 1/2 g(x (j 1)m, X j ) + o p (1). It follows from the covolutio theorem that a estimator ˆϕ is regular ad efficiet if ad oly if it is asymptotically liear with efficiet ifluece fuctio ( g(x 0, x) = ϑk t ϕ x k 1 l k (x k ϑ k x k 1 ) + u ϕk (x k ϑ k x k 1 ) ). The ier product i (5) decomposes ito m + 1 ier products o R p ad Ū1,..., Ūm. This implies that the gradiet of a fuctioal ϕ of ϱ oly is the same for each submodel i which some or all of the f k are kow. Hece asymptotically we caot estimate ϕ better i these submodels. I this sese, fuctioals ϕ(ϱ) are adaptive with respect to f. Similarly, fuctioals of f k are adaptive with respect to the other parameters. For a q-dimesioal fuctioal ϕ = (ϕ 1,..., ϕ q ) of (ϱ, f), differetiability is uderstood compoetwise. For a estimator ˆϕ of ϕ, asymptotic liearity is also uderstood compoetwise, ad regularity is defied as i (6), ow with L a q-dimesioal radom vector. It is the coveiet to write the gradiet of ϕ as a matrix (T ϕ, U ϕ ) whose s-th row is the gradiet of ϕ s ; so differetiability (5) reads m 1/2 (ϕ(ϱ t, f u ) ϕ(ϱ, f)) T ϕ ϑ D ϑt + E[U ϕ, k (ε k )u k (ε k )] (7) for all (t, u) R p U. The covolutio theorem the says that L is distributed as the covolutio of some radom vector with a ormal radom vector N that has mea vector 0 ad covariace matrix T ϕ ϑ D ϑt ϕ + E[U ϕ, k (ε k )U ϕ, k (ε k )]. Fially, ˆϕ is regular ad efficiet if ad oly if it is asymptotically liear with (q-dimesioal) efficiet ifluece fuctio ( g(x 0, x) = Tϕ ϑ k x k 1 l k (x k ϑ k x k 1 ) + U ϕ, k (x k ϑ k x k 1 ) ).
3 Costructio of efficiet estimators Iferece for Alteratig Time Series 5 Autoregressio parameters. Suppose we wat to estimate ϱ. By adaptivity, the gradiet of the fuctioal ϕ(ϱ) = ϱ is obtaied from (7) as (T ϕ, 0) with T ϕ solvig 1/2 (ϱ t ϱ) = t = T ϕ ϑ D ϑt t R; so T ϕ = ( ϑ D ϑ) 1, ad the efficiet ifluece fuctio is m g(x 0, x) = ( ϑ 1 D ϑ) ϑ k x k 1 l k (x k ϑ k x k 1 ). (8) Hece the asymptotic variace of a efficiet estimator is ( ϑ D ϑ) 1. Followig [Koul ad Schick, 1997], we ca costruct a efficiet estimator of ϱ, with this ifluece fuctio, by the Newto Raphso procedure. This is a oe-step improvemet of a root- cosistet iitial estimator. As iitial estimator of ϱ we ca take e.g. the least squares estimator ϱ, the miimum i ϱ of ( ) 2, Xjm+k ϑ k (ϱ)x jm+k 1 i.e. a solutio of the martigale estimatig equatio A efficiet estimator is the ϑ ( ) k (ϱ)x jm+k 1 Xjm+k ϑ k (ϱ)x jm+k 1 = 0. ˆϱ = ϱ + ( ϑ( ϱ) D ϑ( ϱ)) 1 1 ϑ k ( ϱ)x jm+k 1 lk ( ε jm+k ). Here we have estimated l k by l k = f k / f k with f k a appropriate kerel estimator of f k based o residuals ε jm+k = X jm+k ϑ k ( ϱ)x jm+k 1 for j = 1,...,, ad we have estimated D by pluggig i empirical estimators for γ k = E[X 2 k ] ad J k, γ k = 1 X 2 jm+k, Jk = 1 l 2 k( ε jm+k ). A special case is the alteratig AR(1) model (1) with equal autoregressio parameters ϑ 1 = = ϑ m = ϑ. This is described by the reparametrizatio ϑ(ϑ) = (ϑ,..., ϑ), with ϑ playig the role of ϱ. The ϑ = (1,..., 1) ad
6 Müller et al. ϑ D ϑ = D 0 = E[X1 2 ]J 1 + +E[Xm]J 2 m, ad the efficiet ifluece fuctio (8) reduces to g(x 0, x) = D0 1 x k 1 l k (x k ϑx k 1 ). Hece the asymptotic variace of a efficiet estimator is D0 1. A iitial estimator for ϑ is the least squares estimator ϑ = X t 1 X t / m X 2 t 1, ad a efficiet estimator is the oe-step improvemet ˆϑ = ϑ 1 1 + D 0 X jm+k 1 lk ( ε jm+k ) with ε t = X t ϑx t 1 ad D 0 = γ 1 J1 + + γ m Jm. Iovatio distributios. Suppose we wat to estimate a liear fuctioal ϕ(f k ) = E[h(ε k )] = h(x)f k (x) dx of the iovatio distributio, where h L 2 (f k ). By adaptivity, the gradiet of ϕ is obtaied from (5) as (0, u ϕ ) with u ϕi = 0 for i k ad u ϕk solvig 1/2 (ϕ(f kuk ) ϕ(f k )) = E[h(ε k )u k (ε k )] = E[u ϕk (ε k )u k (ε k )], u k U k ; so u ϕk is the projectio of h oto Ūk, u ϕk (x) = h(x) E[h(ε k )] E[ε kh(ε k )] E[ε 2 k ] x. Hece the efficiet ifluece fuctio is g(x 0, x) = h(x k ϑ k x k 1 ) E[h(ε k )] E[ε kh(ε k )] E[ε 2 k ] (x k ϑ k x k 1 ), ad the asymptotic variace of a efficiet estimator is E[u 2 ϕk(ε k )] = Var h(ε k ) (E[ε kh(ε k )]) 2 E[ε 2 k ]. A efficiet estimator, with ifluece fuctio g, is ˆϕ = 1 h( ε jm+k ) ε jm+kh( ε jm+k ) 1 ε2 jm+k ε jm+k. This requires that h or f k is sufficietly smooth. For appropriate assumptios we refer to [Schick ad Wefelmeyer, 2002]. A alterative to the above additive correctio of a empirical estimator are weighted empirical estimators 1 w jh( ε jm+k ), where the radom weights w j are chose such that w j ε jm+k = 0; see [Owe, 2001] ad [Müller et al., 2005].
4 Equal iovatio desities Iferece for Alteratig Time Series 7 Suppose that the iovatio distributios are kow to be equal, f 1 = = f m = f, say. As i Sectio 2, assume that f is absolutely cotious with fiite Fisher iformatio J = E[l 2 (ε)], where l = f /f. Itroduce perturbatios f u (x) = f(x)(1 + 1/2 u(x)) with u i the space U of bouded measurable fuctios such that E[u(ε)] = 0 ad E[εu(ε)] = 0. The local asymptotic ormality (4) reduces to log dp tu dp = 1/2 m ϑ k tx jm+k 1 l(ε jm+k ) + 1/2 u(ε t ) 1 2 t ϑ D1 ϑt m 2 E[u2 (ε)] + o p (1), where D 1 is the diagoal matrix with etries E[X 2 1 ]J,..., E[X 2 m]j. Let Ū deote the closure of U i L 2(f). A real-valued fuctioal ϕ of (ϱ, f) is differetiable at (ϱ, f) with gradiet (t ϕ, u ϕ ) R p Ū if 1/2 (ϕ(ϱ t, f u ) ϕ(ϱ, f)) t ϕ ϑ D 1 ϑt + me[uϕ (ε)u(ε)] for all (t, u) R p U. The factor m is there because we cout the observatios i blocks of legth m. The ier product o the right-had side decomposes ito two ier products o R p ad o Ū. Hece fuctioals of ϱ or f are adaptive with respect to the other parameter. Autoregressio parameters. reduces to g(x 0, x) = ( ϑ D 1 ϑ) 1 The efficiet ifluece fuctio (8) of ϱ ϑ x k 1 l(x k ϑ k x k 1 ). A efficiet estimator ˆϱ of ϱ is agai obtaied as oe-step improvemet of a root- cosistet iitial estimator ϱ, ˆϱ = ϱ + ( ϑ( ϱ) D1 ϑ( ϱ)) 1 ϑ k ( ϱ)x jm+k 1 l( εjm+k ). Here we ca estimate l by l = f / f with f a kerel estimator based o all residuals ε jm+k = X jm+k ϑ k ( ϱ)x jm+k 1 for j = 1,..., ad k = 1,..., m, ad we ca estimate J by J = 1 m l 2 ( ε t ).
8 Müller et al. Iovatio distributio. Suppose we wat to estimate a liear fuctioal ϕ(f) = E[h(ε)] for h L 2 (f). By adaptivity, the gradiet of ϕ is (0, u ϕ ) with u ϕ Ū solvig 1/2 (ϕ(f u ) ϕ(f)) = E[h(ε)u(ε)] = me[u ϕ (ε)u(ε)], u U; so mu ϕ is the projectio of h oto Ū, mu ϕ (x) = h(x) E[h(ε)] E[εh(ε)] E[ε 2 x. ] Hece the efficiet ifluece fuctio is g(x 0, x) = 1 ( m h(x k ϑ k x k 1 ) E[h(ε)] E[εh(ε)] E[ε 2 ] ad the asymptotic variace of a efficiet estimator is E[g 2 (X 0, X 1 )] = 1 m ) (Var h(ε) (E[εh(ε)])2 E[ε 2. ] A efficiet estimator is obtaied, similarly as i Sectio 3, as ˆϕ = 1 h( ε t ) m m ε th( ε t ) 1 m ε2 t m ) (x k ϑ k x k 1 ), Of course, if both the autoregressio parameters ad the iovatio desities are equal, ϑ 1 = = ϑ m = ϑ ad f 1 = = f m = f, the the alteratig AR(1) model reduces to the usual AR(1) model X t = ϑx t 1 +ε t, where the ε t are idepedet with desity f, ad the sample size is m. ε t. Ackowledgmet. DMS 0405791. Ato Schick was supported i part by NSF Grat Refereces [Bickel et al., 1998]P.J. Bickel, C.A.J. Klaasse, Y. Ritov, ad J.A. Weller. Efficiet ad Adaptive Estimatio for Semiparametric Models. Spriger, New York, 1998. [Koul ad Schick, 1997]H.L. Koul ad A. Schick. Efficiet estimatio i oliear autoregressive time series models. Beroulli, 3:247 277, 1997. [Müller et al., 2005]U.U Müller, A. Schick, ad W. Wefelmeyer. Weighted residualbased desity estimators for oliear autoregressive models. Statistica Siica, 15:177 195, 2005. [Owe, 2001]A.B. Owe. Empirical Likelihood. Chapma & Hall, Lodo, 2001. [Schick ad Wefelmeyer, 2002]A. Schick ad W. Wefelmeyer. Estimatig the iovatio distributio i oliear autoregressive models. Aals of the Istitute of Statistical Mathematics, 54:245 260, 2002.