Dipartimento di Ingegneria e Scienze dell Informazione e Matematica Università dell Aquila, umberto.triacca@univaq.it
Stationary stochastic processes Stationarity is a rather intuitive concept, it means that the statistical properties of the process do not change over time.
Stationary stochastic processes There are two important forms of stationarity: 1 strong stationarity; 2 weak stationarity.
Stationary stochastic processes Strong stationarity concerns the shift-invariance (in time) of its finite-dimensional distributions. Weak stationarity only concerns the shift-invariance (in time) of first and second moments of a process.
Strongly stationary stochastic processes Definition. The process {x t ; t Z} is strongly stationary if F t1 +k,t 2 +k,,t s+k(b 1, b 2,, b s ) = F t1,t 2,,t s (b 1, b 2,, b s ) for any finite set of indices {t 1, t 2,, t s } Z with s Z +, and any k Z. Thus the process {x t ; t Z} is strongly stationary if the joint distibution function of the vector (x t1 +k, x t2 +k,..., x ts+k) is equal with the one of (x t1, x t2,..., x ts ) for for any finite set of indices {t 1, t 2,, t s } Z with s Z +, and any k Z.
Strongly stationary stochastic processes The meaning of the strongly stationarity is that the distribution of a number of random variables of the stochastic process is the same as we shift them along the time index axis.
Strongly stationary stochastic processes If {x t ; t Z} is a strongly stationary process, then x 1, x 2, x 3,... have the same distribution function. (x 1, x 3 ), (x 5, x 7 ), (x 9, x 11 ),... have the same joint distribution function and further (x 1, x 3, x 5 ), (x 7, x 9, x 11 ), (x 13, x 15, x 17 ),... must have the same joint distribution function, and so on.
Strongly stationary stochastic processes If the process is {x t ; t Z} is strongly stationary, then the joint probability distribution function of (x t1, x t2,..., x ts ) is invariant under translation.
iid process An iid process is a strongly stationary process. This follows almost immediate from the definition. Since the random variables x t1 +k, x t2 +k,..., x ts+k are iid, we have that F t1 +k,t 2 +k,,t s+k(b 1, b 2,, b s ) = F (b 1 )F (b 2 ) F (b s ) On the other hand, also the random variables x t1, x t2,..., x ts and hence are iid F t1,t 2,,t s (b 1, b 2,, b s ) = F (b 1 )F (b 2 ) F (b s ). We can conclude that F t1 +k,t 2 +k,,t s+k(b 1, b 2,, b s ) = F t1,t 2,,t s (b 1, b 2,, b s )
iid process Remark. Let {x t ; t Z} be an iid process. We have that the conditional distribution of x T +h given values of (x 1,..., x T ) is P(x T +h b x 1,..., x T ) = P(x T +h b) So the knowledge of the past has no value for predicting the future. An iid process is unpredictable. Example. Under efficient capital market hypothesis, the stock price change is an iid process. This means that the stock price change is unpredictable from previous stock price changes.
Strongly stationary stochastic processes Consider the discrete stochastic process {x t ; t N} where x t = A, with A U (3, 7) (A is uniformly distributed on the interval [3, 7]). This process is of course strongly stationary. Why?
Strongly stationary stochastic processes Consider the discrete stochastic process {x t ; t N} where x t = ta, with A U (3, 7) This process is not strongly stationary Why?
Weakly stationary stochastic processes If the second moment of x t is finite for all t, then the mean E(x t ), the variance var(x t ) = E[(x t E(x t )) 2 ] = E(x 2 t ) (E(x t )) 2 and the covariance cov(x t1, x t2 ) = E[(x t1 E(x t1 ))(x t2 E(x t2 ))] are finite for all t, t 1 and t 2. Why? Hint: Use the Cauchy-Schwarz inequality cov(x, y) 2 var(x)var(y)
Weakly stationary stochastic processes Definition The process {x t ; t Z} is weakly stationary, or covariance-stationary if 1 the second moment of x t is finite for all t, that is E x t 2 < for all t 2 the first moment of x t is independent of t, that is E(x t ) = µ t 3 the cross moment E(x t1 x t2 ) depends only on t 1 t 2, that is cov(x t1, x t2 ) = cov(x t1 +h, x t2 +h) t 1, t 2, h
Weakly stationary stochastic processes Thus a stochastic process is covariance-stationary if 1 it has the same mean value, µ, at all time points; 2 it has the same variance, γ 0, at all time points; and 3 the covariance between the values at any two time points, t, t k, depend only on k, the difference between the two times, and not on the location of the points along the time axis.
Weakly stationary stochastic processes An important example of covariance-stochastic process is the so-called white noise process. Definition. A stochastic process {u t ; t Z} in which the random variables u t, t = 0 ± 1, ±2... are such that 1 E(u t ) = 0 t 2 Var(u t ) = σ 2 u < t 3 Cov(u t, u t k ) = 0 t, k is called white noise with mean 0 and variance σ 2 u, written u t WN(0, σ 2 u). First condition establishes that the expectation is always constant and equal to zero. Second condition establishes that variance is constant. Third condition establishes that the variables of the process are uncorrelated for all lags. If the random variables u t are independently and identically distributed with mean 0 and variance σ 2 u then we will write u t IID(0, σ 2 u)
White Noise process Figure shows a possible realization of an IID(0,1) process. Figure : A realization of an IID(0,1).
Random Walk process An important example of weakly non-stationary stochastic processes is the following. Let {y t ; t = 0, 1, 2,...} be a stochastic processs where y 0 = δ < and y t = y t 1 + u t for t = 1,2,..., with u t WN(0, σ 2 u). This process is called random walk.
Random Walk process The mean of y t is given by E(y t ) = δ and its variance is Var(y t ) = tσu 2 Thus a random walk is not weakly stationary process.
Random Walk process Figure shows a possible realization of a random walk. Figure : A realization of a random walk.
Relation between strong and weak Stationarity First note that finite second moments are not assumed in the definition of strong stationarity, therefore, strong stationarity does not necessarily imply weak stationarity. For example, an iid process with standard Cauchy distribution is strictly stationary but not weak stationary because the second moment of the process is not finite.
Relation between strong and weak Stationarity If the process {x t ; t Z} is strongly stationary and has finite second moment, then {x t ; t Z} is weakly stationary. PROOF. If the process {x t ; t Z} is strongly stationary, then..., x 1, x 0, x 1,... have the same distribution function and (x t1, x t2 ) and (x t1 +h, x t2 +h) have the same joint distribution function for all t 1, and t 2 and h. Because, by hypothesis, the process {x t ; t Z} has finite second moment, this implies that E(x t ) = µ t cov(x t1, x t2 ) = cov(x t1 +h, x t2 +h) t 1, t 2, h
Relation between strong and weak Stationarity Of course a weakly stationary process is not necessarily strongly stationary. weak stationarity strong stationarity
Relation between strong and weak Stationarity Here we give an example of a weakly stationary stochastic process which is not strictly stationary. Let {x t ; t Z} be a stochastic process defined by { u t if t is even x t = 1 2 (ut 2 1) if t is odd where u t iidn(0, 1). This process is weakly stationary but it is not strictly stationary.
Relation between strong and weak Stationarity We have and E(x t ) = { E(u t ) = 0 1 2 E(ut 2 1) = 0 if t is even if t is odd { var(ut ) = 1 if t is even var(x t ) = 1 2 var(u2 t 1 ) = 1 if t is odd Further, because x t and x t k are independent random variables, we have cov(x t, x t k ) = 0 k Thus, the process x t is weakly stationary. In particular, x t WN(0, 1).
Relation between strong and weak Stationarity Now, we note that P(x t 0) = P(u t 0) = 0.5 for t even and ( ) 1 P (x t 0) = P 2 (ut 2 1) 0 = P ( ut 1 2 1 ) = P ( u t 1 1) = P ( 1 u t 1 1) = 0.6826 for t odd Hence the random variables of the process are not identically distributed. This implies that the process is not strongly stationary
Relation between strong and weak Stationarity There is one important case however in which weak stationarity implies strong stationarity. If {x t ; t Z} is a weakly stationary Gaussian stochastic process, then {x t ; t Z} is strongly stationary. Why?
Relation between strong and weak Stationarity Let {x t ; t Z} be a Gaussian stochastic process. Introducing the vector b = (b 1, b 2,..., b s ) R s, the multidimensional density function of the vector (x t1, x t2,..., x ts ) is ( 1 f t1,t 2,...,t s (b) = (2π) s det(σ (Σ) exp 1 ) 2 (b µ) Σ 1 (b µ). where µ = (E(x t1 ), E(x t2 ),..., E(t s )) and Σ = [ Cov(x ti, x tj ) ]. We note that a multivariate Gaussian distribution is fully characterized by its first two moments.
Relation between strong and weak Stationarity Let {x t ; t Z} be a Gaussian stochastic process. Assume that the process is weakly stationary. If the process i weakly stationary, then 1 E(x t ) = µ t 2 Var(x t ) = γ 0 < t 3 Cov(x t1 +k, x t2 +k) = Cov(x t1, x t2 ) t 1, t 2, k and hence f t1 +k,t 2 +k,...,t s+k(b) = f t1,t 2,...,t s (b) It follows that the joint distibution function of the vector (x t1 +k, x t2 +k,..., x ts+k) is equal with the one of (x t1, x t2,..., x ts ) for for any finite set of indices {t 1, t 2,, t s } Z with s Z +, and any k Z. This implies that the process {x t ; t Z} is strongly stationary.
Relation between strong and weak Stationarity Conversely, if {x t ; t Z} is a Gaussian strongly stationary stochastic process, then it is weakly stationary because it has finite variance. Thus we can conclude that in the case of Gaussian stochastic process, the two definitions of stationarity are equivalent.
White Noise process We note that a white noise process is not necessarily strongly stationary. Let w be a random variable uniformly distributed in the interval (0; 2π). We consider the process {Z t ; t = 1, 2,...} defined by Z t = cos(tw) t = 1, 2,... We have that 1 E(Z t ) = 0 t 2 Var(Z t ) = 1 2 t 3 Cov(Z t, Z t k ) = 0 t, k Thus Z t WN(0,.5). However, it can be shown that is not strongly stationary.