A family, Z,of random vecors : Ω R k defined on a probabiliy space Ω, A,P) is called a saionary process if he mean vecors E E =E M = M k E and he auocovariance marices are independen of. k cov, -h )=E E ) -h E -h ) he auocovariance funcion of a saionary process is defined by Γh)=cov, -h ). Exercise: Show ha Γh)=Γ h). MS A saionary process is called whie noise if is auocovariance funcion saisfies Γh)= h. Since Γ) does no have o be a diagonal marix, any wo componens of whie noise can be correlaed wih each oher conemporaneously. A saionary process is called a firs order auoregressive process or AR) process) if i can be expressed as φ φk ) M = M O M M + M, k 3 k 4 φ φkk 443 ) k 443 k 3 Φ where is whie noise wih mean vecor. Each componen of an AR) process depends no only on lagged values of iself bu also on lagged values of he oher componens.
Subsiuing in an AR) equaion =Φ - + firs Φ - + - for - and hen Φ -3 + 3 for -, gives =ΦΦ - + - )+ =Φ - +Φ - + =Φ Φ -3 + - )+Φ - + =Φ 3-3 +Φ - +Φ - + M m =Φ m -m + Φ j j Suppose ha Φ is diagonalizable, i.e., here is an inverible marix C such ha Λ=C - ΦC is a diagonal marix. I hen follows from Λ=C - ΦC Φ=CΛC - For example, any Hermiian marix a complex square marix ha is equal o is own conjugae ranspose) is diagonalizable. hus, real symmeric marices are diagonalizable. ha Φ =ΦΦ=CΛC - CΛC - =CΛ C - Φ 3 =Φ Φ=CΛ C - CΛC - =CΛ 3 C - hus, M Φ m =CΛ m C -. λ Φ m =C M m λ =C M λ λ m O O m M λ k M m λ k C - C - will vanish as m only if he no necessarily real numbers λ,,λ k have modulus less han. MR
A k k marix Φ is diagonalizable if and only if i has k linearly independen eigenvecors diagonalizaion heorem). Proof: Suppose ha c,,c k are linearly independen eigenvecors wih eigenvalues λ,,λ k. hen If hen λ Φ = c,,c k ) M O M c,,c k ) -, λ k Φ = Φc,,c k )c,,c k ) - Φc,,Φc k ) = Φc,,c k ) = Φc,,Φc k )c,,c k ) - = λ c,,λ k c k )c,,c k ) - λ = c,,c k ) M O M c,,c k ) -. λk = c,,c k ) λ M = λ c,,λ k c k ). O λ M k MD 3
Exercise: Show ha if a marix Φ has differen eigenvalues λ and λ, he corresponding eigenvecors c and c will be linearly independen. Soluion: Since λ and λ are differen, a leas one of hem, say λ, is no equal o zero. Assuming ha he anihesis is valid, we obain c =νc for some ν Φc =νφc, λ c =νλ c, and c = λ λ νc, Since λ and c, c, his is in conradicion wih he λ anihesis. ME Exercise: Find he eigenvalues λ and λ of he marix Φ=. Hin: If c is an eigenvecor wih eigenvalue λ, i.e., or, equivalenly, Φc=λc λiφ)c=, hen he inveribiliy of he marix λiφ would imply ha c=λiφ) - =, which is inconsisen wih he requiremen ha c mus be a non-zero vecor. he eigenvalues can herefore be found by solving he equaion deλiφ)=. MV 4
If Φ is a k k marix, he characerisic polynomial deλiφ) has degree k. According o he fundamenal heorem of algebra i has herefore k complex) roos, if each roo is couned wih is algebraic mulipliciy. Since eigenvecors corresponding o differen eigenvalues are independen, Φ can only be non-diagonalizable if here exiss an eigenvalue wih algebraic mulipliciy m a > and geomeric mulipliciy m g <m a. he geomeric mulipliciy of an eigenvalue is he number of linearly independen eigenvecors wih ha eigenvalue. Exercise: Show ha he marix MN is non-diagonalizable. Φ= 3 Hin: he eigenvecors corresponding o an eigenvalue λ can be found by solving he equaion λiφ)c= for c. he condiion ha all he eigenvalues of Φ are less han in absolue value, i.e., is equivalen o z deφzi), z de z ΦzI)), z dei z Φ), and z deizφ). Remark: If all roos of he polynomial dei-zφ) lie ouside of he uni circle, he sequence Φ, Φ, Φ 3, is absoluely summable and j j converges componenwise) in mean square o. Φ 5
sing lag-operaor noaion he equaion can also be wrien as Φ - = IΦ) =, where IΦ is a marix-valued polynomial. For example, in he bivariae case we have I Φ) = φ = φ φ φ φ φ φ φ φ) φ = φ + φ) A saionary process is called an auoregressive process of order p or ARp) process) if i can be expressed as or, equivalenly, as =Φ - + +Φ p -p + Φ - Φ p -p =, where is whie noise wih mean vecor. sing lag-operaor noaion, he laer equaion can also be wrien as where Φ) =, Φ)=IΦ Φ p p is a marix-valued polynomial. = φ φ ) ) φ φ ) ). 6
e be a general linear process represened by = Ψ j j, where is whie noise wih E = and var )=Σ. We have and E = Ψ je j = Γ k)=cov, -k )=E Ψ j = Ψ r= r j E = Ψ j+ kσψ j. j k)j Ψ -r -j+ k) Since neiher E nor cov, -k ) depend on, he process is weakly saionary. P Ψ j he specral densiies of and are given by f ω)= π f ω)= π e k= iωk iωk e = π k= Γ k)= Σ, π Ψ i k e ω = π k= j+ k Γ k) ΣΨ j 4443 4 Ψ j ΣΨ iωj e Ψ jσ) k= = π Ψ j e -iωj Σ k= = π Ψ jk Ψ jk e iωj-k) j+ k e iωj+k) Ψ j e -iωj Σ Ψ -iωk k e * ) k=. PD 7
A represenaion IΦ) = of an AR) process is called causal if can be expressed in erms of presen and pas shocks, i.e., = j j Is specral densiy is given by f ω)= π j j Φ ) = Φ. j Φ e -iωj Σ iω = π Φe ) I Σ k= k e -iωk Φ ) * iω ) * I Φe ). Exercise: Derive he sum formula PG n Φ j =IΦ n+ )IΦ) for a geomeric series of marices. Hin: Muliply each side of he equaion by IΦ. Remark: Moreover, if all eigenvalues of Φ have modulus less han, we have = j j Φ =IΦ). Exercise: Show ha PV = j j Φ e -iωj =IΦe -iω ), if all eigenvalues of Φ have modulus less han. Analogously, f ω)= π IΦ e -iω Φ p e -iωp ) - Σ IΦ e -iω Φ p e -iωp ) - ) * is he specral densiy of an ARp) process wih causal represenaion IΦ Φ p p ) =. 8
Exercise: Reexamine he relaionship beween changes in he indusrial producion and changes in he duraion of unemploymen wih parameric mehods. Wrie an R funcion for he calculaion of he specral densiy of a vecor auoregressive process. var.spec <- funcionfr,ar.p) { # fr vecor of frequencies # AR.p ARp) model esimaed by R funcion ar nf <- lenghfr); p <- AR.p$order sigma <- AR.p$var.pred; k <- lenghsigma[,]) Id <- diag,nrow=k,ncol=k) # ideniy marix sp <- arraydim=cnf,k,k)) for w in :nf) { A <- Id for l in :p) A <- A-AR.p$ar[l,,]*exp-i*fr[w]*l) A <- solvea) # inverse of A sp[w,,] <- A%*%sigma%*%ConjA)) } reurnsp/*pi)) } Esimae AR model of order p=3. AR.3 <- arxy,order.max=3,aic=f,demean=) # aic=f order is fixed, no seleced auomaically # AR.3$ar: array of dim 3,,) wih AR coefficiens AR.3$ar[,,] # lag Series Series Series.6357.78465 Series -.59957 -.3733489 AR.3$ar[,,] # lag Series Series Series.44655.994 Series -.46 -.7367764 AR.3$ar[3,,] # lag 3 Series Series Series.76967 -.95759 Series -.466969 -.74439 AR.3$var.pred # variance no explained by AR model Series Series Series 4.437e-5-6.444694e-6 Series -6.444694e-6 3.7668e-3 9
Esimae he univariae specral densiies. parmfrow=c,),mar=c,,,)) p <- spec.pgramxy[,],aper=,der=f,fas=f,plo=f) f <- p$freq**pi; plof,p$spec/*pi),ype="l") sp.3 <- var.specf,ar.3); linesf,sp.3[,,],col="red") p <- spec.pgramxy[,],aper=,der=f,fas=f,plo=f) plof,p$spec/*pi),ype="l") linesf,sp.3[,,],col="red") Esimae he cospecrum. parmfrow=c,)) plof,resp.3[,,]),ype="l") ablineh=,ly="dashed") # dashed horizonal line he overall negaive relaionship beween he wo variables is mainly due o he low frequencies.
Esimae he squared coherency and he phase specrum. parmfrow=c,)) plof,modsp.3[,,])^/sp.3[,,]*sp.3[,,]),ype="l") plof,argsp.3[,,]),ype="l") he squared coherency is large a he low frequencies. here he slope of he phase specrum is approximaely, which indicaes ha changes in he duraion of unemploymen lag wo monhs behind changes in indusrial producion.
A saionary process is called an auoregressive moving average process of order p,q) or ARMAp,q) process) if i can be expressed as =Φ - + +Φ p -p + +Θ - + +Θ q -q or, equivalenly, as Φ - Φ p -p = +Θ - + +Θ q -q, where is whie noise wih mean vecor. sing lag-operaor noaion, he laer equaion can also be wrien as where and Φ) =Θ), Φ)=IΦ Φ p p Θ)=I+Θ + +Θ q q are marix-valued polynomials. An ARMAp,) process is an ARp) process. An ARMA,q) process is also called a moving average process of order q or MAq) process). he ARMAp,q) equaion IΦ Φ p p ) = I+Θ + +Θ q q ) is said o be causal if z deizφ z p Φ p ). I is said o be inverible if z dei+zθ+ +z q Θ q ). Exercise: Show ha he bivariae AR) process 4 ) ) = is causal and inverible. PC
3 Exercise: Show ha he bivariae MA) process = + 3 ) ) + ) ) is causal and inverible. PI Exercise: Show ha he bivariae ARMA,) process ) ) = + 3 5 ) ) is causal and inverible. PA I does no make sense o esimae he parameer marices Φ,,Φ p,θ,,θ q, and Σ of an ARMAp,q) process if hey are no unique. o ensure idenifiabiliy in he univariae case, where Φ) and Θ) are jus scalar polynomials, we mus require, in addiion o causaliy and inveribiliy, ha Φz) and Θz) have no common zeros. For example, he equaion 4 ) =+ ) can be wrien more parsimoniously as ) =, because he polynomials 4 z =+ z) z) and + z have a common zero.
In he mulivariae case, he marix-valued polynomials Φz) and Θz) can have a common lef facor even if deφz)) and deθz)) have no common zero. o avoid he difficulies involved in he idenificaion of mulivariae ARMA processes, many ime series analyss use only mulivariae AR models for he modeling of mulivariae ime series. Exercise: Show ha he equaion P φ + θ ) ) can be wrien more parsimoniously as φ = θ + ) ) = ) ) alhough he polynomials and have no common zero. φ + θ deφz))=dei z) θ deθz))=dei+ z) Hin: Muliply boh Φz) and Θz) by Θ - θz z)=. Remark: he inverse of he marix-valued polynomial Θz) is also a marix-valued polynomial. Is deerminan is a consan unequal o zero. Such a marix-valued polynomial is called unimodular. 4