{ ϕ(f )Xt, 1 t n p, X t, n p + 1 t n, { 0, t=1, W t = E(W t W s, 1 s t 1), 2 t n.

Transcription

1 Innovations, Sufficient Statistics, And Maximum Lielihood In ARM A Models Georgi N. Boshnaov Institute of Mathematics, Bulgarian Academy of Sciences 1. Introduction. Let X t, t Z} be a zero mean ARMA(p, q) process, ϕ(b)x t = θ(b)ϵ t. Here ϵ t, t Z} is an i.i.d. sequence of gaussian N(0, σ 2 ) random variables, the polynomials ϕ(z) = 1 ϕ 1 z ϕ p z p, and θ(z) = 1 + θ 1 z + + θ q z q, have no common factors and their zeros are in the regions z > 1 and z 1, respectively. Let n be an integer, n > max(p, q), and X = (X 1,..., X n ). It is nown that when q 0 the distribution of X does not admit any sufficient statistic whose dimension is independent of n (see Arato (1961)). Dicinson (1982) showed that if the moving average parameters are fixed then there exists a sufficient statistic whose dimension does not depend on n. The proof of Dicinson is in fact an existence one, though in principle it shows how to compute that statistic. In this paper we give a new expression for the lielihood function of an ARMA process. The main term in this expression is quadratic in the autoregressive parameters. It s coefficients are functions of the moving average parameters only. Explicit formulas for the sufficient statistic (in the case of fixed moving average parameters) are obtained from the lielihood function. The derivation is based on the innovations algorithm (Brocwell, Davis (1987)). We apply it to the ARMA case in a new way. The idea is to transform the sample with the autoregressive operator and begin the innovations algorithm from the side of the new autocovariance matrix which does not depend on ϕ. We apply the autoregressive operator in the forward direction. Only the lower right corner (with size max(p, q) max(p, q)) of the covariance matrix of the transformed sample depends on the autoregressive parameters. Thus, roughly speaing, only the last max(p, q) steps in the innovations algorithm depend on them. 2. Notations and definitions. Let us define W t = ϕ(f )Xt, 1 t n p, X t, n p + 1 t n, Ŵ t = 0, t=1, E(W t W s, 1 s t 1), 2 t n.

2 Here F is the forward shift operator (F X t = X t+1 ). We will use the following notations, along with those in the introduction: W = (W 1,..., W n ), Ŵ = (Ŵ1,..., Ŵn), ϕ = (ϕ 1,..., ϕ p ), θ = (θ 1,..., θ q ) ; v t = var(w t+1 Ŵt+1), r t = v t /σ 2 (r t does not depend on σ 2 ), t = 0,..., n 1; Γ - the covariance matrix of W with entries Γ(t, s) = E(W t W s ); (X,..., X +p 1 ), 1 n p, Y = (0,..., 0), n p + 1 n, min(t, q), 1 t min(n 1, n p + q), s(t) = t n + p, n p + q + 1 t n 1. It is easy to see that R q (t s), s, t = 1,..., n p, R x (t s), n p + 1 s, t n, Γ(t, s) = R c (t s), 1 t n p, n p + 1 s n, R c (s t), 1 s n p, n p + 1 t n, where R x (t s) = E(X t X s ); R c (t s) = E(W t X s ), 1 t n p, n p + 1 s n; and R q (.) is the autocovariance function of a MA(q) process with the same θ(z) and σ 2. Remar. E(W t X s ) = 0 when s t > q, 1 t n p, n p + 1 s n. R q () = 0 for > 0. Therefore, when q p, Γ is a band matrix with band 2q + 1. If q < p then Γ is a band matrix having a p p bloc in the lower right corner. Moreover the entries Γ(t, s) of Γ do not depend on ϕ when 1 s, t n p. Thus, the number of the elements of Γ which depend on ϕ, does not depend on n. 3. Results. Lemma 1 is an innovations algorithm application which taes into account the special pattern of Γ. Note that the sums in Lemma 1 involve at most max(q, p 1) terms. Lemma 1. (Innovations algorithm for W ) The following relations hold: s(t) v 0 = Γ(1, 1), Ŵ 1 = 0, Ŵ t+1 = θ ti (W t+1 i Ŵt+1 i), 1 t n 1, v t = Γ(t + 1, t + 1) t 1 i=t s(t) θ 2 tt iv i, 1 t n 1,

3 where θ tt = 0, = 0, 1,..., t s(t) 1, and θ tt = 1 Γ(t + 1, + 1) v 1 i=t s(t) θ i θ tt i v i, = t s(t),..., t 1. Theorem 1. W i Ŵi = α i + β i ϕ, where β i is a p-dimensional vector, α i is a scalar, α 1 = X 1, β 1 = Y 2, α t = X t θ t 1i α t i, β t = Y t+1 + θ t 1i β t i, 2 t n. Corollary 1. α i and β i, i = 1,..., n p, do not depend on ϕ and σ 2. Theorem 2. a) the log-lielihood function of X is given by l(x) = n 2 log(2π) n 2 log(σ2 ) l 1 (X) l 2 (X), where l 1 (X) = i=0 log(r i ) + 1 2σ 2 (W i Ŵi) 2 /r i, b) l 1 (X) = 1 2 l 2 (X) = 1 2 n 1 i= log(r i ) + 1 2σ 2 n i=+1 (W i Ŵi) 2 /r i, 1 i=0 log(r i ) + S 1 (X)/(2σ 2 ), where S 1 (X) = A + B ϕ + ϕ Cϕ, A is a scalar, B is p 1, C is a p p matrix. A,B,C depend only on θ and X by the relations: A = αi 2 /r i, B = 2 α i β i/r i, C = β i β i/r i. Theorem 3. The quantities A, B, C, α i, β i, i = n p,..., n p q + 1, X +1,..., X n, form a sufficient statistic for ϕ, σ 2 when θ is fixed. The dimension of this statistic is at most p 2 /2 + (5/2 + q)p + q Proofs. Lemma 1. follows from Proposition of Brocwell and Davis (1987) and the following properties of the coefficients in the innovations algorithm. For clarity they are formulated as lemmas.

4 θ (2) Let R, R 1, and R 2 be nonsingular covariance matrices. Let v 0, v (1) 0, v(2) 0, θ,j, θ (1),j,,j, v, v (1), v(2), = 1,..., t, j = 0,..., 1, be the coefficients in the innovations algorithm for R, R 1 and R 2, respectively (see Brocwell and Davis (1987)). Lemma 2. If R 1 is proportional to R 2 (R 1 = cr 2, c > 0) then θ (1),j = cv (2), for all, j. v (1) Then = θ(2),j, and Lemma 3. Let t and m (t > m) be such that R(t + 1, ) = 0 for = 1,..., t m. θ tt = 0 = 0, 1,..., t m 1, θ tt (t m) = θ tm = R(t + 1, t m + 1)/v, = t m, ( ) 1 θ tt = R(t + 1, + 1) θ i θ tt i v i /v, = t m + 1,..., t 1, v t = R(t + 1, t + 1) t 1 i=t m i=t m θ 2 tt iv i. In the last two formulas the number of the summands is not greater than m. We omit the proofs of Lemma 2 and Lemma 3 since they are straightforward. To prove Theorem 1 we note that W 1 Ŵ1 = W 1 = X 1 p ϕ i X 1+i = X 1 Y 2ϕ = α 1 + β 1ϕ, where α 1 = X 1, β 1 = Y 2. Let t [2, n]. Suppose that W i Ŵi = α i +β i ϕ, i = 1,..., t 1. Then by Lemma 1 W t Ŵt = W t θ t 1i (W t i Ŵt i) = = X t Y t+1ϕ θ t 1i (α t i + β t iϕ) = = X t By induction we obtain Theorem 1. θ t 1i α t i Y t+1 + θ t 1i β t i ϕ. Corollary 1 is obtained now from Theorem 1, Lemma 2, and the properties of Γ. Part a) of Theorem 2 holds because the Jacobian of the transformation from X to W is unity, and the two vectors are gaussian.

5 We get part b) of Theorem 2. from the equations (W i Ŵi) 2 /r i = (α i + β iϕ) 2 /r i = (αi 2 + 2α i β iϕ + ϕ β i β iϕ)/r i. The assertion of Theorem 3 follows from Theorem 1, Corollary 1, and Theorem Applications. Some problems lead to the estimation of ARM A models with fixed moving average parameters. For example, let θ(z) be a polynomial of degree q with all roots on the unit circle, Y t = θ(b)x t, H0 = (Y t is AR(p) ), H 1 = (Y t is ARMA(p, q), with moving average operator θ()). By testing the hypothesis H 0 against H 1 one can decide whether to include the nonstationary operator θ() in the autoregressive model. The application to the case of unnown moving average parameters is also possible, at least for large n. The sufficient statistic can be used to speed up the maximization of the lielihood function along the autoregressive coordinates. REFERENCES 1. Arato, M. On the Sufficient Statistics for Stationary Gaussian Random Processes. Theory Prob.Appl. 6, 1961, Brocwell, P.J., R.A.Davis, Time Series: Theory and Methods. Springer Series in Statistics, Dicinson, B.W., Sufficient Statistics for Stationary Discrete-time Gaussian Random Processes. J. Time Series Anal. 3, 3, 1982,