A generalization of the Burg s algorithm to periodically correlated time series

Transcription

1 A generalizaion of he Burg s algorihm o periodically correlaed ime series Georgi N. Boshnakov Insiue of Mahemaics, Bulgarian Academy of Sciences ABSTRACT In his paper periodically correlaed processes are considered. A generalizaion of he Burg s algorihm o his class of processes is obained by minimising a sum of squared forward-backward residuals. The resuling filer is sable. The algorihm given here is differen from he Burg-ype algorihm proposed previously by Sakai. Also, his algorihm is obained by replacing he expecaions in he recursion formulas by heir sample esimaors, no by minimising a sum of squared forward-backward residuals. A more symmeric form of he Sakai s Levinson-ype recursion formulas is also given. Of some ineres may be also he saisical derivaion of hese formulas. Georgi Boshnakov Insiue of Mahemaics Acad. G.Bonchev sr., Sofia BULGARIA FAX: (+359 2) boshnako@bgearn.bine

2 1 Inroducion. 2 Forward and backward residuals Le {X } be a zero-mean d-periodically correlaed process wih auocovariance funcion R s = EX X s. This means ha he ineger d is such ha R s = R +d,s+d, for any pair, s. Le ρ k be he parial correlaion coefficien beween X and X k, afer removing he dependence on he inermediae X s, X 1,..., X k+1. For k = 0 we se ρ 0 = R,. A zero-mean d-periodically correlaed process whose auocovariance funcion R s = 0 when s, is called periodic whie noise. Definiion 1 The process {X } is said o be periodic auoregression PAR(p 1,...,p d ) wih period d > 0, and orders p = (p 1,..., p d ) if i is periodically correlaed and if for every, p X ϕ i X i = ε, (1) where {ε } is periodic whie noise P W N(0, σ 2, d) and all parameers are d- periodic oo, i.e. p +d = p, ϕ +di = ϕ i, σε 2 +d = σε 2. The noaion M(X...) is used o denoe he projecion of X on he space spanned by he variables o he righ of he verical bar. In he Gaussian case his coinsides wih he condiional expecaion, oherwise M(X...) is he bes linear predicor. 1

3 We pu a ha on X for he forward predicors and coefficiens, and a ilde for he backward ones. The forward residuals are denoed by ε, he backward ones by a. We pu an addiional upper index o he various quaniies o show on how many X s is based he corresponding predicor. Wih hese noaions we have X (k) = k i X i, and X (k) = k i X +i, X = M(X X 1,..., X k ) + ε (k) = X = M(X X 1,..., X k+1 ) + ε (k 1) = X k = M(X k X k+1,..., X ) + a (k) k = X k = M(X k X k+1,..., X 1 ) + a (k 1) k = X (k) X (k 1) + ε (k) + ε (k 1) X (k) k + a(k) k X (k 1) k + a (k 1) k Therefore, X (k) = = X (k 1) X (k 1) + M(ε (k 1) X 1,..., X k ) + M(ε (k 1) X 1,..., X k+1, a (k 1) k ) The leers σ and v are used o denoe he variances of he forward and backward residuals, respecively. More precisely σ 2 (k) var(x (k) X ), v 2 (k) (k) var(x X ). 2

4 The spaces spanned on {X 1,..., X k+1 } and {a (k 1) }, respecively, are orhogonal. By consrucion ε (k 1) is orhogonal o he firs of hem. Therefore, M(ε (k 1) X 1,..., X k+1, a (k 1) k ) = M(ε (k 1) = M(ε (k 1) a (k 1) k ) = δa (k 1) k for some δ. Therefore, k X 1,..., X k+1 ) + M(ε (k 1) a (k 1) k ) X (k) = M( X (k 1) X 1,..., X k+1 ) + M(ε (k 1) a (k 1) k ) = = X (k 1) X (k 1) + M(ε (k 1) a (k 1) k ) + δa (k 1) k (2) This shows ha ε (k) + δa (k 1) k = ε (k 1). The random variables on he lefhand side of his equaion are uncorrelaed. Therefore, σ 2 (k) = σ 2 (k 1) δ 2 v 2 (k 1) k Expanding he condiional expecaions and δa (k 1) k in (2) we obain k i X i = = k 1 k 1 ϕ (k 1) k 1 i X i + δx k δ ϕ (k 1) k 1 i X i δ γ (k 1) ki X k+i γ (k 1) kk i X k+k i + δx k 3

5 Comparing he coefficiens a X i we see ha δ = k i = ϕ (k 1) i k γ(k 1) kk i, i = 1,..., k 1 σ 2 (k) = σ 2 (k 1) ( k )2 v 2 (k 1) k In paricular, he firs equaion shows ha k regression coefficien of ε (k 1) can be inerpreed as he on a (k 1). Therefore, i is given by he formula k )σ(k 1) k = corr(ε (k 1), a (k 1) k k σ (k 1) = ρ k k, = E(ε(k 1) a (k 1) k ) v 2(k 1) k where ρ k is he parial correlaion coefficien beween X and X k (see he beginning of his secion). Applying he same argumens in he reverse ime direcion we obain ha kk is he regression coefficien of a(k 1) k on ε (k 1). Therefore, and kk = corr(ε (k 1) k = ρ k σ (k 1), a (k 1) k ) v(k 1) k σ (k 1) = E(ε(k 1) a (k 1) k ) σ 2(k 1) ki = γ (k 1) ki kk ϕ(k 1) k i, i = 1,..., k 1 v 2 (k) k = v 2 (k 1) k ( kk )2 σ 2 (k 1), 4

6 where v 2 (k) k is he variance of a (k) k. Therefore, Eε (k 1) a (k 1) k = kk σ2 (k 1) = k v2 (k 1) k (3) From he above formulas we have he following simple, bu imporan Lemma. Lemma 1 k = 0 if and only if γ(k) kk = 0. Muliplying he equaions for k and γ(k) kk we obain ρ 2 k = k γ(k) kk (4) The equaions for he residuals can be wrien also in he form ε (k) (k) = X X = ε (k 1) k a(k 1) k a (k) (k) k = X X k = a(k 1) k kk ε(k 1). Muliplying he firs equaion by ε (k 1) and ε (k) and using (3) we obain Eε (k) ε (k 1) = σ 2 (k 1) σ 2 (k) Therefore, = Eε (k 1) σ 2 (k) ε (k) k Ea(k 1) k ε(k 1) = σ 2 (k 1) (1 k γ(k) kk ). = σ 2 (k 1) (1 k γ(k) kk ) Similarly, for he variances of he backward residuals we ge. k = v 2 (k 1) k (1 k γ(k) kk ). v 2 (k) From hese equaions and (4) follows ha he condiion k γ(k) kk < 1 or equivalenly ρ k < 1 is necessary and sufficien for hese formulas o give 5

7 non-negaive σ 2 (k) definie, or he filer o be sable. and v 2 (k) k, i.e. he correlaion marix o be non-negaive Noe also ha he quoien σ 2 (k) /v 2 (k) k hrough he lower order variances only. = σ 2 (k 1) /v 2 (k 1) k We summarise he recursion equaions in he following Lemma. is expressible Lemma 2 The k-h order parameers can be deermined from he (k 1)-h order ones and ρ 1k,..., ρ dk by he following formulas: (i) for k = 0, (ii) for k 1, σ (k 1) k = ρ k k σ 2 (0) = v 2 (0) = ρ 0, i = ϕ (k 1) i k γ(k 1) kk i, i = 1,..., k 1 σ 2 (k) = (1 ρ 2 k)σ 2 (k 1), = 1,..., d k kk = ρ k σ (k 1) ki = γ (k 1) ki kk ϕ(k 1) k i, i = 1,..., k 1 v 2 (k) k = (1 ρ 2 k)v 2 (k 1) k, = 1,..., d From his Lemma we can see ha he parial correlaions deermine compleely he parameers and herefore he periodic auoregression model. Noe also he symmery of he formulas wih respec o he forward and backward direcion. 6..

8 Lemma 3 An equivalen paramerizaion of an periodic auoregression model PAR(p 1,...,p d ) is hrough he se of parial correlaions ρ 0 > 0, ρ k, = 1,..., d, k = 1,..., p. The model is sable if and only if ρ k < 1, for all k 1 and all. As in he saionary case he sable condiion is expressed simply by independen resricions on ρ k. Lemma 2 can be used also wih differen orders for he differen ime periods by puing ρ k = 0 when k > p. This is jusified by Lemma 1. 3 Generalizaion of he Burg s algorihm Le (X 1,..., X n ) be aken from a d-periodically correlaed process. Le us consider he following likelihood-like weighed forward-backward sum of squared residuals. where S 0 = n =k+1 2 ε(k) σ 2(k) S 0 can be rewrien as S 0 = d 1 σ 2(k) + a(k) k v 2(k) k 2 n = =k+1 δ k = σ(k) v (k) k [ n k i ] d l=0 1 σ 2(k) = σ(k 1) k. ( ε (k) 2 + δ 2 k a (k) k2 ), ( ε (k) 2 k+i+ld + δ 2 k+i+ld,k a (k) i+ld2 ), 7

9 I can be easily seen from he periodic propery of {X } ha var(ε (k) k+i+ld ), var(a (k) i+ld ), and δ k+i+ld,k, do no depend on he inner summaion index l and are equal o σ 2 (k) k+i, v 2 (k) i, and δ k+i,k, respecively. We are looking for such k-h order parameers which minimise S 0 under he consrain ha he lower order parameers are fixed. From he previous subsecion we know ha he parial correlaions give a paramerizaion of he model and hey are independen of each oher. We will see below ha he minimisaion of S (k) i reduces o minimisaion wih respec o ρ k, which implies ha he minimisaion of S 0 reduces o independen minimisaion of d non overlapping sums. In view of his remark we can define he objecive funcion in a slighly more general way as S (k k 1) (ρ 1k,..., ρ dk ) = d where c 2 i are arbirary posiive weighs and S (k k 1) i (ρ 1k,..., ρ dk ) = [ n k i ] d l=0 c 2 i S (k k 1) i (ρ 1k,..., ρ dk ), ( ε (k) 2 k+i+ld + δ 2 k+i,k a (k) i+ld2 ). The noaion sresses he fac ha we consider his funcion condiionally on he (k 1)-h order parameers being fixed. Proposiion 4 i.e. (i) The funcion S (k k 1) i (ρ 1k,..., ρ dk ) depends on ρ ik only, S (k k 1) i (ρ 1k,..., ρ dk ) = S (k k 1) i (ρ ik ). 8

10 (ii) min S (k k 1) (ρ 1k,..., ρ dk ) = ρ 1k,...,ρ dk d c 2 i min S (k k 1) ρ i (ρ ik ) ik (iii) min ρik S (k k 1) i (ρ ik ) = S (k k 1) i (ˆρ ik ), where ˆρ ik = 2δ ik S ikεa S ikεε + δik 2 S, i = 1,..., d ikaa S ikεε = S ikaa = S ikεa = [ n k i ] d ε (k 1) 2 k+i+ld l=0 [ n k i ] d a (k 1) 2 i+ld l=0 [ n k i ] d ε (k 1) k+i+ld a(k 1) i+ld l=0 (iv) ˆρ ik 1. (v) ˆρ ik = 1 if and only if ε (k 1) k+i+ld = δ ika (k 1) k i i+ld, l = 0,..., [n ] d This resul serves as a basis for he following algorihm. The algorihm esimaes he parial correlaion coefficiens, he variances of he residuals, and (if necessary) he coefficiens of he periodic auoregressive models from he following inpu: he observed sample (X 1,..., X n ), he period and he maximal orders K 1,..., K d. 9

11 Algorihm 1 (Periodic Burg Algorihm) (i k is o be replaced by i k + d when i k 0) K = max (K 1,..., K d ) σ 2 (0) j = v 2 (0) j = 1 [ n j d ] + 1 [ n j d ] i=0 X 2 j+id, ε (0) = a (0) = X, = 1,..., n j = 1,..., d for (k=1,...,k) do { Compue S εε, S aa, S εa, using equaions (5). 2δ ik S ikεa ˆρ ik = S ikεε + δik 2 S if k K i ikaa, i = 1,..., d 0 if k > K i σ 2 (k) i v 2 (k) i k = (1 ˆρ 2 ik)σ 2 (k 1) i = (1 ˆρ 2 ik)v 2 (k 1) i k = ρ σ (k 1) k k k kk = ρ k σ (k 1) for (j=1,...,k-1) do { ij = ϕ (k 1) ij ik γ(k 1) i kk j, i kj = γ(k 1) i kj i kk ϕ(k 1) ik j i = 1,..., d } for (=k+1,...,n) do { ε (k) = ε (k 1) k a(k 1) k 10

12 a (k) k = a(k 1) k kk ε(k 1) } } The asympoic properies of he esimaors are he same as he ones obained from he periodic Yule-Walker equaions [5], [6]. In he saionary case he formula for ρ reduces o ˆρ k = 2S ϵa (S ϵϵ + S aa ), (5) since δ is equal o one in ha case. This coincides wih he Burg s equaions. Algorihm 1 is a sraighforward generalizaion of he Burg s algorihm o he periodic case. In he saionary case he forward and backward coefficiens and he residual variances coincide so ha he reflecion coefficiens coincide wih he corresponding parial correlaions. As noed by Jones [2] he esimae (5) is he (approximae) maximum likelihood esimaor of a correlaion coefficien for Gaussian independen pairs ε and a wih mean zero and unknown bu equal variances, while for compleely unknown variances he esimaor is S εa Sεε S aa. (6) Our formula for ˆρ k can be inerpreed as an esimae of a correlaion coefficien when he raio of he variances is known. 11

13 This algorihm has a highly parallel srucure and admis efficien realisaion. Noe ha he Burg-ype algorihm given by Sakai [6] is differen from ours and is obained by replacing he mahemaical expecaions in he formulas for he reflecion coefficiens by heir sample esimaors, no by minimisaion of he forward-backward sum of squares. His esimaors can be inerpreed as esimaors for a correlaion coefficien when he variances are unknown (he equaion (6)). This algorihm can be used also for mulivariae specral esimaion. I seems wih lower compuaional coss han he available mulivariae Burglike algorihms, compare i e.g. wih he mehods given in [2]. Anoher advanage of he algorihm which sems from he scalar periodic represenaion of a mulivariae process is ha i may give more parsimonious represenaion. The number of parameers grows as kd wih he order of he model, while for he mulivariae auoregression - as kd 2. A disadvanage of he algorihm when applied o he mulivariae problem is ha when applied direcly he resul will depend on he order in which he mulivariae vecor is arranged for he scalar represenaion. An exac correspondence beween a mulivariae auoregression model of order P and a periodic model wih orders p 1,..., p d can be obained by seing p i = P d + i 1, i = 1,..., d. Algorihm 1 can be used o esimae such models. The orders of he auoregression can be seleced by sandard crieria, such 12

14 as AIC. As is he case wih he parial correlaion, he AIC urns ou o spli ino independen AIC crierions for each season individually (see [6], [5] for deails). References [1] E.G. Gladishev. Periodically correlaed random sequences. Sovie Mah., 2: , [2] R.H. Jones. Mulivariae auoregressive esimaion using residuals. In D.F. Findley, edior, Applied ime series analysis, pages Lecure noes in Economics and Mah. Sysems, [3] R.H. Jones and W.M. Brelsford. Time series wih periodic srucure. Biomerika, 54: , [4] H.J. Newon. Using periodic auoregression for muliple specral esimaion. Technomerics, 24(2): , [5] M. Pagano. On periodic and muliple auoregression. Ann.Sais., 6: , [6] H. Sakai. Circular laice filering using Pagano s mehod. IEEE Trans.Acous.Speech Signal Process., 30(2): ,