PREDICTION AND NONGAUSSIAN AUTOREGRESSIVE STATIONARY SEQUENCES 1. Murray Rosenblatt University of California, San Diego

Transcription

1 PREDICTION AND NONGAUSSIAN AUTOREGRESSIVE STATIONARY SEQUENCES 1 Murray Rosenblatt University of California, San Diego Abstract The object of this paper is to show that under certain auxiliary assumptions a stationary autoregressive sequence has a best predictor in mean square that is linear if and only if the sequence is minimum phase or is Gaussian when all moments are finite 1 Introduction We consider a stationary autoregressive sequence, that is, a stationary sequence x t satisfying the system of equations 11 x t φ 1 x t 1 φ p x t p =ξ t, t =, 1,0,1, with the ξ t s independent identically distributed, the φ i s real and Eξ t 0,Eξ 2 t = σ 2 >0 Let 12 φz =1 φ 1 z φ p z p The system of equations is satisfied by a strictly stationary sequence which is uniquely determined if and only if φz has no roots of absolute value 1 In [4] a simple result of the type considered in this paper was established for a first order autoregressive scheme x t satisfying 13 x t βx t 1 = ξ t, t =, 1,0,1,, 0< β <1 Clearly the best one-step predictor predicting ahead of x t+1 is the linear predictor βx t However, the best one-step predictor with time reversed for the process 13, Ex t x t+1 is linear if and only if the distribution of x t is Gaussian Let G be the distribution function of ξ t and let F the distribution function of x t It is clear that F satisfies the equation F =G Fβ 1, the asterisk denotes the convolution operation If ϕ is the characteristic function of ξ t,η the characteristic function of x t ητ = ϕβ j τ, 1991 Mathematics Subject Classification Primary 60G25, 62M20; secondary 60G10, 10J10 Key words and phrases Autoregressive sequence, nongaussian, nonminimum phase, nonlinear prediction Research supported by ONR Grant N J1371 1

2 2 MURRAY ROSENBLATT and φτ 1,τ 2 =Eexpiτ 1 x 1 + iτ 2 x 0 the joint characteristic function of x 1 and x 0, the following relation is satisfied: 14 φ τ1 0,τ 2 = 1 β η τ 2 1 β ϕ τ 2 ηβτ 2 = expiτ 2 xiex 1 x 0 = x df x Let Gx = x udgu The relation 14 implies that Ex 1 x 0 = x is given by 1 β x 1 { G F β 1 dx/f dx β A related problem for heavy-tailed distributions is considered in [2] Factor the polynomial 12, φz =φ + zφ z φ + z =1 θ 1 z θ r z r 0 for z 1, φ z=1 θ r+1 z θ p z s 0 for z 1 and r, s 0,r+s=p Given that m 1,,m r,m r+1,,m p are the p zeros of φz let m i > 1,i=1,,r and m i < 1,i=r+1,,p Then φ + z = r i=1 1 m 1 i z, φ z= p i=r+1 1 m 1 i z The autoregressive sequence 11 is called minimum phase if r = p and nonminimum phase otherwise If the sequence is minimum phase, clearly one can write 15 x t = α j ξ t j 16 φz 1 = α j z j and the α j s decay to zero exponentially fast as j The relations 15 and 16 imply that the σ-algebras generated by {ξ j,j t and {x j,j t are the same This implies that in the minimum phase case the best predictor in mean square of x t+1 given x j,j t, is linear and given by x t = α j ξ t+1 j Our object is to show that in the nonminimum phase nongaussian case, the best predictor in mean square of x t+1 given x j,j t, is nonlinear if all moments of ξ t are finite and the roots m i,i = r +1,,p, are distinct In Section 2 we show

3 STATIONARY AUTOREGRESSIVE SEQUENCES 3 that the stationary solution of 11 is pth order Markovian This implies that the best one-step predictor in mean square in terms of the past is a function of the p preceding variables That the solution of 11 is pth order Markovian is obvious in the minimum phase case since the solution is causal, implying that ξ t is independent of the past of the x process, that is x t 1,x t 2, In the nonminimum phase case the x t process is noncausal and so ξ t is not independent of the past of the x process The Markovian property of the x process is used in Section 3 the principal result on the nonlinearity of the best predictor in mean square of x t+1 given x j,j t, is derived in the nonminimum phase nongaussian case when all the moments of the ξ t are finite and the roots m i,i = r +1,,p are distinct One should note that the first order autoregressive scheme with time reversed discussed in this section is not minimum phase 2 The Markov Property Our object in this section is to show that the stationary autoregressive sequence is pth order Markovian, whether it is minimum phase or not Part of the argument parallels one given in [1] The argument is carried out in the case r, s > 0, since it is obvious otherwise Introduce the causal and purely noncausal sequences U t = φ Bx t, V t = φ + Bx t, with B the one-step backshift symbol, that is, Bx t = x t 1 We then have Let us also note that we have U t = α j ξ t j, V t = β j ξ t+j j=s φ + z 1 = α j z j, φ z 1 = β j z j x t = 21 φz 1 = j= ψ j ξ t j, j= ψ j z j We shall carry through the argument assuming the existence of positive density functions However, essentially the same argument can be carried through without this assumption using a more elaborate notation Let the density function of the ξ random variables be g The random variables U l,l t, are independent of V l,l t s+1, and so the joint probability density function of U 1,,U n,v n s+1,,v n is { n h U U 1,, U r g U t θ 1 U t 1 θ r U t r h V V n s+1,,v n t=r+1 j=s

4 4 MURRAY ROSENBLATT h U and h V are the joint probability density functions of U 1,,U r and V n s+1,,v n respectively Consider the linear transformation T n given by U 1 U s U s+1 U n V n s+1 V n U 1 U U 1 s x s+1 θ r+1 x s θ p x 1 = U =T s n x 1 x n θ r+1 x n 1 θ p x n s x n s+1 θ 1 x n s θ r x n s+1 r x n x n θ 1 x n 1 θ r x n r Using this transformation one can see that the joint density of U 1,,U s,x 1,,x n is h U Ũ1,,Ũr { n { p t=p+1 t=r+1 g Ũt θ 1 Ũ t 1 θ r Ũ t r g x t φ 1 x t 1 φ p x t p h V φ + Bx n s+1,,φ + Bx n dett n { Ul if l s Ũ l = x l θ r+1 x l 1 θ p x l s if l>s If s>0, ln dett n ln θ p n p Let us compute the conditional density of x n,x n 1,,x n p given x n d,x n 1 d,,x 1,U s,,u 1 The one-step d =1 conditional density is given by gx n φ 1 x n 1 φ p x n p h Vφ + Bx n s+1,, φ + Bx n h V φ + Bx n s,,φ + Bx n 1 as if 1 <d p+ 1, one obtains { d 1 gx n u φ 1 x n 1 u φ p x n u p u=0 h Vφ + Bx n s+1,,φ + Bx n h V φ + Bx n d s+1,,φ + Bx n d dett n dett n d dett n dett n 1, If d>p+ 1 the conditional probability density is { n gx t φ 1 x t 1 φ p x t p dx n d+1 dx n p 1 t=n d+1 h V φ + Bx n s+1,,φ + Bx n h V φ + Bx n d s+1,,φ + Bx n d dett n dett n d

5 STATIONARY AUTOREGRESSIVE SEQUENCES 5 Notice that in all these cases the conditional probability density depends on x n d,x n d 1,,x 1,U s,,u 1 only through x n d,,x n d p+1 However, this implies that the conditional probability density of x n,x n 1,,x n p given x n d,x n 1 d,,x 1 is the same by a standard argument using conditional expectations The argument is that if f is integrable, B and A are σ-algebras, then if Ef B,A=his B measurable, it follows that Ef B = EEf B,A B = Eh B = h Thus {X n is a sequence that is pth order Markovian 3 A functional equation for the characteristic function The characteristic function of x k is clearly ηt = ϕψ k t The joint characteristic function of the random variables x s,x s+1,,x 0 is { [ ητ s,τ s 1,,τ 0 =E exp i ] s τ l x l = l=0 as the joint characteristic function of x s,x s+1,,x 1 is It is clear that ητ s,τ s 1,,τ 1 = s ϕ τ l ψ k l ητ s,,τ 1,τ 0 τ0=0 = η τ0 τ 2,,τ 1,0 τ 0 s = ix 0 exp i τ l x l df x s,,x 1,x 0 =i Ex 0 x 1,,x s exp i s ϕ τ l ψ k l, l=0 s τ l x l df x s,,x 1, F x s,,x 1 is the joint distribution function of x s,,x 1 In the case of the pth order autoregressive sequence x t, since the sequence is a Markov process of order p, it is sufficient in considering the best one-step predictor in mean square to consider s = p, since the one-step predictor of x 0 given the whole past will depend only on the p immediately preceding random variables Now log ητ p,,τ 1,τ 0 τ0=0 = η τ 0 τ p,,τ 1,0 τ 0 ητ p,,τ 1 / = ψ k ϕ τ l ψ k l ϕ τ l ψ k l while τ j log ητ p,,τ 1 = ψ k j ϕ / τ l ψ k l ϕ τ l ψ k l

6 6 MURRAY ROSENBLATT for some neighborhood of the origin τ p,, τ 1 ε, ε > 0 If the best predictor is linear we must have 31 η τ0 τ p,,τ 1,0 = b j η τj τ p,,,τ 1, the b j s are the coefficients of the best linear predictor of x 0 in mean square This is in turn equivalent to 32 ψ k x 0 = b j x j b l ψ k l h τ j ψ k j =0 hτ =ϕ τ / ϕτ for τ 1,,τ p such that ητ p,,τ 1 0 That 31 is equivalent to 32 follows from the fact that η τj τ p,,τ 0 { E x 2 j 1/2 and that the ψ k tend to zero exponentially as k The equation 32 is similar to the type of functional equation taken up in [3] The ψ k are the coefficients in the Laurent expansion of φz 1 The b l can be read off from the polynomial with constant coefficient positive, having the same absolute value as φz when z = e iλ and with all its zeros outside the unit disc Let 1 φ z = 1 s z s φ z Notice that the roots of φ z are the inverses of the roots of φ z Thus the polynomial ζz =φ + zφ z has all its roots outside the unit disc and has the same absolute value as φz Notice that the coefficients ψ k p b lψ k l are those in the Laurent expansion of ζzφz 1 = φ zφ z 1 If the sequence is minimum phase this function is 1 and so the coefficients are 0 for k 0 and 1 when k = 0 Further, in the minimum phase case, ψ k = 0 for k<0 The relation 32 is automatically satisfied in the minimum phase case whatever the distribution G as long as Eξ t = 0 and Eξt 2 < However, we had already seen this before using another argument Proposition 1 Consider the stationary solution x t of the system of equations 11, the ξ t are independent, identically distributed with Eξ t 0,Eξ t = σ 2 < and characteristic polynomial φ [clearly φz 0if z =1] Then the best one-step predictor for x t is linear if and only if 32 holds the ψ k s are given by 21 and the b l s the coefficients of the best linear predictor It is of some interest to essentially characterize the sequence γ k = ψ k b l ψ k l, k =, 1,0,1,

7 STATIONARY AUTOREGRESSIVE SEQUENCES 7 The generating function of the ψ k is φz 1 [see21] Since the coefficients b l correspond to best linear one-step predictor in mean square 1 b l z l = cφz p i=r+1 with c a nonzero constant It then follows that p 33 γz = γ k z k = c Since i=r+1 { 1 mi zm 1 i 1 m 1 i z { 1 mi zm 1 i 1 m 1 i z m 1 1 zm1 m 1 z 1 = m +m 2 1 when m < 1, if follows that m j 1 z j γ k = 0 for k>0 Theorem 1 Consider the stationary autoregressive sequence {x t satisfying 11 and assume that the random variables ξ t have all moments finite Further let the sequence be nonminimum phase with all the zeros m i,i=r+1,,p simple Then if the best one-step predictor is linear, the ξ distribution is Gaussian Suppose that the ξ distribution is nongaussian There are then an infinite number of nonzero cumulants µ a 0,a>2, of the ξ distribution Also ψ k = j=r+1 α j m k j, k > 0 for some coefficients α j 0,j = r +1,,p If µ a+1 0 for some a 2, the relation 32 implies that 34 γ k ψ k l1 ψ k la =0, l 1,,l a =1,,p k=0 For the ath order partial derivative of the expression in 32 with respect to τ l1,,τ la at τ l1 = = τ la =0,i a+1 µ a+1 a! multiplied by the expression on the left of 34 The equations 34 can be rewritten j 1,,j a=r+1 α j1 α ja m l1 j 1 m la j a γ k m j1 m ja k =0, l 1,,l a =1,,p Consider the set of equations obtained by letting l 1,,l a = 1,,s The matrix of this set of equations is { M =M j,l = α j1 α ja m l1 j 1 m la j a, j =j 1,,j a,l=l 1,,,l a,j 1,,j a =r+1,,p,l 1,,l a =1,,s The determinant of this matrix is p u=r+1 α a u times the a th power of the Vandermonde determinant m l j ; j = r +1,,p,,,s k=0

8 8 MURRAY ROSENBLATT Since the determinant is nonzero we must have γ m j1 m ja 1 = γ k m j1 m ja k =0, k=0 j 1,,j a =r+1,,p These are too many zeros for the function γz and so we must have µ a+1 = 0 Since this holds for any a 2, the ξ distribution must be Gaussian NonGaussian nonminimum phase autoregressive sequences arise naturally when considering transects of certain classes of random fields References [1] F Breidt, R Davis, K S Lii and M Rosenblatt, Maximum likelihood estimation for noncausal autoregressive processes, J Mult Anal 36, 1991, [2] S Cambanis and I Fakhre-Zaker, On prediction of heavy-tailed autoregressive sequences: regression versus best linear prediction, Technical report No 383, Center for Stochastic Processes, University of North Carolina [3] B Ramchandran and Ka-Sing Lau, Functional Equations in Probability Theory, Academic Press, 1991 [4] M Rosenblatt, A note on prediction and an autoregressive sequence, Stochastic Processes S Cambanis, J Ghosh, R Karandikar and P Sen, eds, 1993, pp Department of Mathematics University of California, San Diego La Jolla, California 92093