Lesson 9: Autoregressive-Moving Average (ARMA) models

Transcription

1 Lesson 9: Autoregressive-Moving Average (ARMA) models Dipartimento di Ingegneria e Scienze dell Informazione e Matematica Università dell Aquila, umberto.triacca@ec.univaq.it

2 Introduction We have seen that in the class of stationary, zero mean, Gaussian processes the probabilistic structure of a stochastic process is completly characterized by the autocovariance function.

3 Autocovariance function Stationary, zero mean, Gaussian process DGP γ x (k) x 1,..., x T

4 Introduction However, in general, to know the autocovariance function means to know a sequence composed by an infinite number of elements. We have to estimate a infinite number of parameters from observed data. This mission is impossible γ x (0), γ x (1), γ x (2),...,

5 Introduction We introduce a very important class of stochastic processes, which autocovariance functions depend on a finite number of unknown parameters: the class of the AutoregRessive Moving Average (ARMA) processes.

6 Autoregressive-Moving Average (ARMA) models Definition. The process {x t ; t Z} is an autoregressive moving average process of order (p, q), denoted with if x t ARMA(p, q), x t φ 1 x t 1... φ p x t p = u t + θ 1 u t θ q u t q t Z, where u t WN(0, σ 2 u), and φ 1,..., φ p, θ 1,..., θ q are p + q constants and the polynomials and have no common factors. φ(z) = 1 φ 1 z... φ p z p θ(z) = 1 + θ 1 z... + θ q z q

7 Autoregressive-Moving Average (ARMA) models For q = 0 the process reduces to an autoregressive process of order p, denoted with x t AR(p), x t φ 1 x t 1... φ p x t p = u t t Z, For p = 0 to a moving average process of order q, denoted with x t MA(q) x t = u t + θ 1 u t θ q u t q t Z,

8 An example of Autoregressive-Moving Average (ARMA) process The process {x t ; t Z} defined by x t = 0.3x t 1 + u t + 0.7u t 1 t Z, where u t WN(0, σ 2 u), is an ARMA(1,1) process. Here φ(z) = 1 0.3z and θ(z) = z.

9 An example of Autoregressive-Moving Average (ARMA) process A realizzation of the ARMA(1,1) process x t = 0.3x t 1 + u t + 0.7u t 1 is presented in the following figure.

10 An example of Autoregressive (AR) process The process {x t ; t Z} defined by x t = 0.7x t 1 0.5x t 1 + u t t Z, where u t WN(0, σ 2 u), is an AR(2) process. Here φ(z) = 1 0.7z + 0.5z 2

11 An example of Autoregressive (AR) process A realizzation of the AR(2) process x t = 0.7x t 1 0.5x t 2 + u t is presented in the following figure.

12 An example of Moving Average (MA) process The process {x t ; t Z} defined by x t = u t + 0.7u t 1 t Z, where u t WN(0, σ 2 u), is an MA(1) process. Here θ(z) = z

13 An example of Moving Average (MA) process A realizzation of the MA(1) process x t = u t + 0.7u t 1 is presented in the following figure.

14 An example of over-parameterization Consider the process {x t ; t Z} defined by x t = x t x t 2 + u t 0.7u t 1 t Z, where u t WN(0, σ 2 u). This process looks like an ARMA(2,1) process but it is not an ARMA(2,1) process.

15 An example of over-parameterization Here and φ(z) = 1 z z 2 = (1 0.7z)(1 0.3z) θ(z) = 1 0.7z We note that both polynomials have a common factor, namely 1 0.7z. Discarding the common factor in each leaves and φ (z) = 1 0.3z θ (z) = 1. Thus the process is an AR(1) process, defined by x t = 0.3x t 1 + u t

16 Causal Autoregressive-Moving Average (ARMA) models Definition. An ARMA(p, q) process {x t ; t Z} is causal (strictly, a causal function of {u t ; t Z}) if there exists constants ψ 0, ψ 1,... such that ψ j < j=0 and x t = ψ j u t j t. j=0

17 Autoregressive-Moving Average (ARMA) models Here, it is important to clarify the meaning of equality x t = ψ j u t j t j=0 It means that lim E n ( x t n j=0 ) 2 ψ j u t j = 0. The equality is defined in terms of a limit in the quadratic mean.

18 Autoregressive-Moving Average (ARMA) models The following two theorems provide, respectively, a characterization of the of causality and stationarity of an ARMA(p, q) process.

19 Autoregressive-Moving Average (ARMA) models Theorem. An ARMA(p, q) process {x t ; t Z} is causal if and only if φ(z) = 1 φ 1 z... φ p z p 0 for all z 1. Theorem. An ARMA(p, q) process {x t ; t Z} is stationary if and only if φ(z) = 1 φ 1 z... φ p z p 0 for all z = 1.

20 Autoregressive-Moving Average (ARMA) models The causality and the stationarity of an ARMA process depend entirely on the autoregressive parameters and not on the moving-average ones.

21 Autoregressive-Moving Average (ARMA) models Further, we note that if an ARMA(p, q) process is causal, then is stationary, but stationarity does not imply causality.

22 Autoregressive-Moving Average (ARMA) models Consider, for example, the following AR(1) process: x t = 3x t 1 + u t where u t WN(0, σu). 2 We have that φ(z) = 1 3z 0 for all z = 1. and hence the process is stationary, but non causal since φ(z) = 1 3z = 0 for z = 1/3.

23 Autoregressive-Moving Average (ARMA) models An important result: There is a one-to-one correspondence between the parameters of a causal ARMA(p,q) process and the autocovariance function.

24 Autoregressive-Moving Average (ARMA) models It is important to underline that if we consider the set of autocorrelation functions there is not a one-to-one correspondence between the parameters of a causal ARMA(p,q) process and the autocorrelation function.

25 Autoregressive-Moving Average (ARMA) models Consider the following two MA(1) processes. x t = u t + θu t 1 where u t WN(0, σ 2 u), with θ < 1 and y t = u t + 1 θ u t 1 where u t WN(0, σ 2 u). Since θ 1 + θ 2 = 1/θ 1 + (1/θ) 2, we have that both processes share the same autocorrelation function. Thus it cannot be used to distinguish between the two parametrizations.

26 Autoregressive-Moving Average (ARMA) models This example shows that an MA(1)-process is not uniquely determined by its autocorrelation function. There is an identification problem with the MA(1) models. In general, (if all roots of θ(z) = 0 are real) there can be 2 q different MA(q) processes with the same autocorrelation function.

27 Autoregressive-Moving Average (ARMA) models Definition. An ARMA(p, q) process {x t ; t Z} is invertible (strictly, an invertible function of {u t ; t Z}) if there exists constants π 0, π 1,... such that π j < j=0 and u t = π j x t j t. j=0

28 Autoregressive-Moving Average (ARMA) models The following theorem provides a necessary and sufficient condition for the invertibility. Theorem. An ARMA(p, q) process {x t ; t Z} is invertible if and only if θ(z) = 1 + θ 1 z θ q z q 0 for all z 1.

29 Autoregressive-Moving Average (ARMA) models We note that an AR(p) process is always invertible, even if it is non-stationary, while an MA(q) process is always stationary, even if it is non-invertible.

30 Autoregressive-Moving Average (ARMA) models The invertibility can be used in order to ensure the identifiability of MA processes.

31 Autoregressive-Moving Average (ARMA) models In general, (if all roots of θ(z) = 0 are real) there can be 2 q different MA(q) processes with the same autocorrelation function, but only one of these is invertible.

32 Conclusion In the class of the mean-zero causal and invertible Gaussian ARMA processes there is a one-to-one correspondence between the family of the finite dimensional distributions of the process and the finite parametric representation of process.

33 Conclusion In the class of the mean-zero causal and invertible Gaussian ARMA processes the probabilistic properties of the process are completely characterized by the finite set of parameters { φ1, φ 2,..., φ p, θ 1, θ 2,..., θ q, σ 2 u} Now, we have to estimate a finite number (p + q + 1) of parameters from observed data. This mission is possible.

34 Conclusion Zero-mean causal invertible Gaussian ARMA process DGP x 1,..., x T {φ 1, φ 2,..., φ p, θ 1, θ 2,..., θ q, σ 2 u}