Statistics of stochastic processes

Transcription

1 Introduction Statistics of stochastic processes Generally statistics is performed on observations y 1,..., y n assumed to be realizations of independent random variables Y 1,..., Y n. 14 settembre / 31

2 Introduction Statistics of stochastic processes Generally statistics is performed on observations y 1,..., y n assumed to be realizations of independent random variables Y 1,..., Y n. In statistics of stochastic processes (= time series analysis) we will assume y 1,..., y n to be realizations of a stochastic process... Y 1,..., Y n,... with some rules for dependence. 14 settembre / 31

3 Introduction Aims of time series analysis Drawing inference from available data, but first we need to find an appropriate model. Once a model has been selected: provide a compact and correct description of data (trend, seasonal and random terms) Remarks 14 settembre / 31

4 Introduction Aims of time series analysis Drawing inference from available data, but first we need to find an appropriate model. Once a model has been selected: provide a compact and correct description of data (trend, seasonal and random terms) adjust data (filtering, missing values) [separating noise from signal] Remarks 14 settembre / 31

5 Introduction Aims of time series analysis Drawing inference from available data, but first we need to find an appropriate model. Once a model has been selected: provide a compact and correct description of data (trend, seasonal and random terms) adjust data (filtering, missing values) [separating noise from signal] test hypotheses (increasing trend? influence of factors) understand causes Remarks 14 settembre / 31

6 Introduction Aims of time series analysis Drawing inference from available data, but first we need to find an appropriate model. Once a model has been selected: provide a compact and correct description of data (trend, seasonal and random terms) adjust data (filtering, missing values) [separating noise from signal] test hypotheses (increasing trend? influence of factors) understand causes Remarks predict future values 14 settembre / 31

7 Introduction Aims of time series analysis Drawing inference from available data, but first we need to find an appropriate model. Once a model has been selected: provide a compact and correct description of data (trend, seasonal and random terms) adjust data (filtering, missing values) [separating noise from signal] test hypotheses (increasing trend? influence of factors) understand causes Remarks predict future values - Random terms generally described as a stationary process. 14 settembre / 31

8 Introduction Aims of time series analysis Drawing inference from available data, but first we need to find an appropriate model. Once a model has been selected: provide a compact and correct description of data (trend, seasonal and random terms) adjust data (filtering, missing values) [separating noise from signal] test hypotheses (increasing trend? influence of factors) understand causes Remarks predict future values - Random terms generally described as a stationary process. - Linear analysis (additive decomposition of trend, seasonal and stationary process) 14 settembre / 31

9 Introduction Stationary process Definition A stochastic process {X t } t Z is (strictly) stationary if the joint distribution of (X t1, X t2,..., X tk ) is equal to the distribution of (X t1 +h, X t2 +h,..., X tk +h) k N, h Z, t 1, t 2,..., t k Z. In particular, if a stationary stochastic process has finite second moment, then E(X t ) and Cov(X t, X t+h ) do not depend on t. 14 settembre / 31

10 Introduction Stationary process. 2 Linear time series analysis looks only at second-order properties. Then Definition A stochastic process {X t } t Z is stationary if it is in L 2 and E(X t ) = µ Cov(X t, X t+h ) = γ(h). 14 settembre / 31

11 Introduction Stationary process. 2 Linear time series analysis looks only at second-order properties. Then Definition A stochastic process {X t } t Z is stationary if it is in L 2 and E(X t ) = µ Cov(X t, X t+h ) = γ(h). If a Gaussian process is stationary, then it is strictly stationary. 14 settembre / 31

12 Introduction Stationary process. 2 Linear time series analysis looks only at second-order properties. Then Definition A stochastic process {X t } t Z is stationary if it is in L 2 and E(X t ) = µ Cov(X t, X t+h ) = γ(h). If a Gaussian process is stationary, then it is strictly stationary. A Gaussian process is such that all finite-dimensional distributions are multivariate normal. 14 settembre / 31

13 Introduction Reminders on multivariate normal Definition Y = (Y 1,..., Y n ) is multivariate normal if, a R n, a t Y is a univariate normal. 14 settembre / 31

14 Introduction Reminders on multivariate normal Definition Y = (Y 1,..., Y n ) is multivariate normal if, a R n, a t Y is a univariate normal. Equivalently, Y is multivariate normal there exists b R n A (n m) matrix, X = (X 1,..., X m ) independent standard normal r.v. such that Y = AX + b. 14 settembre / 31

15 Introduction Reminders on multivariate normal Definition Y = (Y 1,..., Y n ) is multivariate normal if, a R n, a t Y is a univariate normal. Equivalently, Y is multivariate normal there exists b R n A (n m) matrix, X = (X 1,..., X m ) independent standard normal r.v. such that Y = AX + b. = E(Y ) = b, Cov(Y ) = AA t, i.e. Y N(b, AA t ). 14 settembre / 31

16 Introduction Reminders on multivariate normal Definition Y = (Y 1,..., Y n ) is multivariate normal if, a R n, a t Y is a univariate normal. Equivalently, Y is multivariate normal there exists b R n A (n m) matrix, X = (X 1,..., X m ) independent standard normal r.v. such that Y = AX + b. = E(Y ) = b, Cov(Y ) = AA t, i.e. Y N(b, AA t ). Alternative characterization via characteristic function. 14 settembre / 31

17 Introduction Reminders on multivariate normal Definition Y = (Y 1,..., Y n ) is multivariate normal if, a R n, a t Y is a univariate normal. Equivalently, Y is multivariate normal there exists b R n A (n m) matrix, X = (X 1,..., X m ) independent standard normal r.v. such that Y = AX + b. = E(Y ) = b, Cov(Y ) = AA t, i.e. Y N(b, AA t ). Alternative characterization via characteristic function. If Cov(Y ) = S positive definite (i.e. invertible), Y N(µ, S) has density f Y (y) = (2π) n/2 S 1/2 exp{ (y µ) t S 1 (y µ)/2}. (non-singular distribution) 14 settembre / 31

18 Introduction Gaussian processes Definition A process {X t } is Gaussian, if for any n > 0 and any (t 1,..., t n ) the vector X = (X t1,..., X tn ) has a non-singular multivariate normal distribution. 14 settembre / 31

19 Introduction Gaussian processes Definition A process {X t } is Gaussian, if for any n > 0 and any (t 1,..., t n ) the vector X = (X t1,..., X tn ) has a non-singular multivariate normal distribution. Then let µ = (µ t1,..., µ tn ) = E(X) and Cov(X) = Γ = {γ(t i, t j ), i, j = 1... n}. X has density function { g(x, µ, Γ) = (2π) n/2 Γ 1/2 exp 1 } 2 Γ 1 (x µ), x µ. 14 settembre / 31

20 Introduction Gaussian processes Definition A process {X t } is Gaussian, if for any n > 0 and any (t 1,..., t n ) the vector X = (X t1,..., X tn ) has a non-singular multivariate normal distribution. Then let µ = (µ t1,..., µ tn ) = E(X) and Cov(X) = Γ = {γ(t i, t j ), i, j = 1... n}. X has density function { g(x, µ, Γ) = (2π) n/2 Γ 1/2 exp 1 } 2 Γ 1 (x µ), x µ. {X t } is (weakly) stationary if µ t µ and γ(t i, t j ) = γ( t i t j ); then is also strictly stationary, as the distribution depends only on µ and Γ. 14 settembre / 31

21 Introduction Gaussian processes Definition A process {X t } is Gaussian, if for any n > 0 and any (t 1,..., t n ) the vector X = (X t1,..., X tn ) has a non-singular multivariate normal distribution. Then let µ = (µ t1,..., µ tn ) = E(X) and Cov(X) = Γ = {γ(t i, t j ), i, j = 1... n}. X has density function { g(x, µ, Γ) = (2π) n/2 Γ 1/2 exp 1 } 2 Γ 1 (x µ), x µ. {X t } is (weakly) stationary if µ t µ and γ(t i, t j ) = γ( t i t j ); then is also strictly stationary, as the distribution depends only on µ and Γ. Linear time series analysis is very well suited for Gaussian processes; less so for non-gaussian ones. 14 settembre / 31

22 Introduction Hilbert spaces Many time series problems can be solved using Hilbert space theory. Indeed space L 2 (Ω) is a Hilbert space with X, Y = E(XY ), X Y 2 = E( X Y 2 ). Restricting to the 0-mean subspace X, Y = Cov(X, Y ). 14 settembre / 31

23 Introduction Detrending data Often data do not appeat as arising from stationary processes. Estimating trend, and then study residuals (differences from trend) smoothing polynomial (esp. line) fitting Study differenced series In all cases, trasformations may be useful More systematic model fitting in the future. 14 settembre / 31

24 Johnson & Johnson quarterly earnings J & J Earnings per Share data 3 points smoothing 5 points smoothing Time 14 settembre / 31

25 Johnson & Johnson data: deviations from trend Deviations from moving average Time 14 settembre / 31

26 Johnson & Johnson data: deviations in log-scale Deviations (in log scale) from moving average Time 14 settembre / 31

27 Sunspots data sunspots year 14 settembre / 31

28 Sunspots data : square-root transformation sunspots year 14 settembre / 31

29 PanAm international air passengers Passengers (1000's) Time 14 settembre / 31

30 PanAm yearly data Annual air passengers aggregate(ap) Time 14 settembre / 31

31 PanAm monthly variation Seasonal component in air passengers Month 14 settembre / 31

32 Level of Lake Huron Level of lake Huron ft Time 14 settembre / 31

33 Lake Huron level: deviations from trend Deviations from trend in level of lake Huron ft Time 14 settembre / 31

34 sales of red wine in Australia Red wine sales in Australia kilolitres Time 14 settembre / 31

35 Deviation from trend in wine sales Deviations from trend in sales of red wine kilolitres Time 14 settembre / 31

36 PanAm monthly variation Seasonal variation in wine sales (AUS) settembre / 31

37 Global temperature data Global temperature data Anomalies from mean Monthly averages Yearly averages Time 14 settembre / 31

38 Global temperature: recent years and trend Global temperatures (regression line in blue) Anomalies Time 14 settembre / 31

39 Measles data in England Measles in England cases per biweek year 14 settembre / 31

40 EEG data from a subject with epilepsy EEG time (arbitrary unit) 14 settembre / 31

41 De-trend and de-seasonalize (period T = 2q) yearly average: m t = 1 T ( 1 2 x t q + q 1 j= (q 1) x t+j x t+q ). 14 settembre / 31

42 De-trend and de-seasonalize (period T = 2q) yearly average: m t = 1 T ( seasonal deviation: w k = 1 n 1 2 x t q + n 1 j=0 q 1 j= (q 1) x t+j x t+q ) (x jt +k m jt +k ), k = 1... T.. 14 settembre / 31

43 De-trend and de-seasonalize (period T = 2q) yearly average: m t = 1 T ( seasonal deviation: w k = 1 n 1 2 x t q + n 1 j=0 seasonal component: ŝ k = w k 1 T ŝ t = ŝ t [ t 1 T ]T, t > T. q 1 j= (q 1) x t+j x t+q ) (x jt +k m jt +k ), k = 1... T. T w i, k = 1... T. i=1. 14 settembre / 31

44 De-trend and de-seasonalize (period T = 2q) yearly average: m t = 1 T ( seasonal deviation: w k = 1 n 1 2 x t q + n 1 j=0 seasonal component: ŝ k = w k 1 T ŝ t = ŝ t [ t 1 T ]T, t > T. deseasonalized data d t = x t ŝ t. q 1 j= (q 1) x t+j x t+q ) (x jt +k m jt +k ), k = 1... T. T w i, k = 1... T. i=1. 14 settembre / 31

45 De-trend and de-seasonalize (period T = 2q) yearly average: m t = 1 T ( seasonal deviation: w k = 1 n 1 2 x t q + n 1 j=0 seasonal component: ŝ k = w k 1 T ŝ t = ŝ t [ t 1 T ]T, t > T. deseasonalized data d t = x t ŝ t. q 1 j= (q 1) x t+j x t+q ) (x jt +k m jt +k ), k = 1... T. T w i, k = 1... T. i=1 ˆm t trend component on deseasonalized data.. 14 settembre / 31

46 De-trend and de-seasonalize (period T = 2q) yearly average: m t = 1 T ( seasonal deviation: w k = 1 n 1 2 x t q + n 1 j=0 seasonal component: ŝ k = w k 1 T ŝ t = ŝ t [ t 1 T ]T, t > T. deseasonalized data d t = x t ŝ t. q 1 j= (q 1) x t+j x t+q ) (x jt +k m jt +k ), k = 1... T. T w i, k = 1... T. i=1 ˆm t trend component on deseasonalized data. Ŷ t = x t ˆm t ŝ t random component.. 14 settembre / 31

47 De-trend and de-seasonalize (period T = 2q) yearly average: m t = 1 T ( seasonal deviation: w k = 1 n 1 2 x t q + n 1 j=0 seasonal component: ŝ k = w k 1 T ŝ t = ŝ t [ t 1 T ]T, t > T. deseasonalized data d t = x t ŝ t. q 1 j= (q 1) x t+j x t+q ) (x jt +k m jt +k ), k = 1... T. T w i, k = 1... T. i=1 ˆm t trend component on deseasonalized data. Ŷ t = x t ˆm t ŝ t random component. Otherwise, difference data: T X t := X t X t T. T X t are de-seasonalized; then a trend can be eliminated from these.. 14 settembre / 31

48 Autocovariance and autocorrelation functions If a process {X t } is stationary, γ(h) := Cov(X t, X t+h ) is the Autocovariance function (ACVF). 14 settembre / 31

49 Autocovariance and autocorrelation functions If a process {X t } is stationary, γ(h) := Cov(X t, X t+h ) is the Autocovariance function (ACVF). Recall the correlation ρ(x, Y ) = Cov(X, Y ) V (X )V (Y ). For a stationary process V (X t ) = V (X t+h ) = γ(0). Hence ρ(h) = ρ(x t, X t+h ) = γ(h) γ(0) is the Autocorrelation function (ACF). 14 settembre / 31

50 Autocovariance and autocorrelation functions If a process {X t } is stationary, γ(h) := Cov(X t, X t+h ) is the Autocovariance function (ACVF). Recall the correlation ρ(x, Y ) = Cov(X, Y ) V (X )V (Y ). For a stationary process V (X t ) = V (X t+h ) = γ(0). Hence ρ(h) = ρ(x t, X t+h ) = γ(h) γ(0) First properties of ACVF: is the Autocorrelation function (ACF). 14 settembre / 31

51 Autocovariance and autocorrelation functions If a process {X t } is stationary, γ(h) := Cov(X t, X t+h ) is the Autocovariance function (ACVF). Recall the correlation ρ(x, Y ) = Cov(X, Y ) V (X )V (Y ). For a stationary process V (X t ) = V (X t+h ) = γ(0). Hence ρ(h) = ρ(x t, X t+h ) = γ(h) γ(0) First properties of ACVF: is the Autocorrelation function (ACF). γ(h) = γ( h) [stationarity = Cov(X t, X t+h ) = Cov(X t h, X t )] 14 settembre / 31

52 Autocovariance and autocorrelation functions If a process {X t } is stationary, γ(h) := Cov(X t, X t+h ) is the Autocovariance function (ACVF). Recall the correlation ρ(x, Y ) = Cov(X, Y ) V (X )V (Y ). For a stationary process V (X t ) = V (X t+h ) = γ(0). Hence ρ(h) = ρ(x t, X t+h ) = γ(h) γ(0) First properties of ACVF: is the Autocorrelation function (ACF). γ(h) = γ( h) [stationarity = Cov(X t, X t+h ) = Cov(X t h, X t )] γ(h) γ(0) [as ρ(x, Y ) 1] 14 settembre / 31

53 Simple stationary processes and their ACVF IID(0, σ 2 ): {X t } t Z independent and identically distributed r. v. with E(X t ) = 0, V(X t ) = σ 2 : γ(0) = σ 2, γ(h) = 0 for h > settembre / 31

54 Simple stationary processes and their ACVF IID(0, σ 2 ): {X t } t Z independent and identically distributed r. v. with E(X t ) = 0, V(X t ) = σ 2 : γ(0) = σ 2, γ(h) = 0 for h > 0. WN(0, σ 2 ) [white noise] {X t } t Z uncorrelated random variables with mean 0 and variance σ 2 : γ(0) = σ 2, γ(h) = 0 for h > settembre / 31

55 Simple stationary processes and their ACVF IID(0, σ 2 ): {X t } t Z independent and identically distributed r. v. with E(X t ) = 0, V(X t ) = σ 2 : γ(0) = σ 2, γ(h) = 0 for h > 0. WN(0, σ 2 ) [white noise] {X t } t Z uncorrelated random variables with mean 0 and variance σ 2 : γ(0) = σ 2, γ(h) = 0 for h > 0. WN(0, σ 2 ) need not be independent. For instance if {Z t } t Z are IID and N(0,1) [normal r.v.], then { Z t t odd X t = (Zt 1 2 1)/ is WN(0, 1) but not IID(0, 1). 2 t even It is not IID, since (e.g.) X 1 and X 2 are obviously not independent. Left for exercise that X t is WN. 14 settembre / 31

56 Simple stationary processes and their ACVF IID(0, σ 2 ): {X t } t Z independent and identically distributed r. v. with E(X t ) = 0, V(X t ) = σ 2 : γ(0) = σ 2, γ(h) = 0 for h > 0. WN(0, σ 2 ) [white noise] {X t } t Z uncorrelated random variables with mean 0 and variance σ 2 : γ(0) = σ 2, γ(h) = 0 for h > 0. WN(0, σ 2 ) need not be independent. For instance if {Z t } t Z are IID and N(0,1) [normal r.v.], then { Z t t odd X t = (Zt 1 2 1)/ is WN(0, 1) but not IID(0, 1). 2 t even It is not IID, since (e.g.) X 1 and X 2 are obviously not independent. Left for exercise that X t is WN. Less contrived examples of {X t } t Z WN but not IID will be seen later in the course. 14 settembre / 31

57 Moving average processes and their ACVF. 2 MA(1): moving average {X t } t Z is MA(1) if X t = Z t + ϑz t 1, t Z where ϑ R, {Z t } WN(0, σ 2 ). A simple computation: γ(0) = σ 2 (1 + ϑ 2 ), γ(1) = ϑσ 2, γ(h) = 0 for h > settembre / 31

58 Moving average processes and their ACVF. 2 MA(1): moving average {X t } t Z is MA(1) if X t = Z t + ϑz t 1, t Z where ϑ R, {Z t } WN(0, σ 2 ). A simple computation: γ(0) = σ 2 (1 + ϑ 2 ), γ(1) = ϑσ 2, γ(h) = 0 for h > 1. Similarly {X t } t Z MA(q) if X t = Z t + ϑ 1 Z t 1 + ϑ q Z t q, t Z, with ϑ 1,..., ϑ q R, {Z t } WN(0, σ 2 ). Another simple computation leads to γ(h) = 0 for h > q. 14 settembre / 31

59 AutoRegressive processes AR(1) [AutoRegressive] {X t } t Z is AR(1) if is stationary and X t = φx t 1 + Z t, t Z where φ R, {Z t } WN(0, σ 2 ). (1) 14 settembre / 31

60 AutoRegressive processes AR(1) [AutoRegressive] {X t } t Z is AR(1) if is stationary and X t = φx t 1 + Z t, t Z where φ R, {Z t } WN(0, σ 2 ). (1) (1) is an (infinite set of) equation. It is not obvious that a stationary process exists satisfying them (this will be discussed later). 14 settembre / 31

61 AutoRegressive processes AR(1) [AutoRegressive] {X t } t Z is AR(1) if is stationary and X t = φx t 1 + Z t, t Z where φ R, {Z t } WN(0, σ 2 ). (1) (1) is an (infinite set of) equation. It is not obvious that a stationary process exists satisfying them (this will be discussed later). We are not saying {X t } t N is the Markov chain defined through X t = φx t 1 + Z t, t > 0 with X 0 some prescribed r.v. 14 settembre / 31

62 AutoRegressive processes AR(1) [AutoRegressive] {X t } t Z is AR(1) if is stationary and X t = φx t 1 + Z t, t Z where φ R, {Z t } WN(0, σ 2 ). (1) (1) is an (infinite set of) equation. It is not obvious that a stationary process exists satisfying them (this will be discussed later). We are not saying {X t } t N is the Markov chain defined through X t = φx t 1 + Z t, t > 0 with X 0 some prescribed r.v. Now, assume a stationary process {X t } t Z exists satisfying (1) and E(X t Z s ) = 0 for t < s (this latter property seems natural as X t should be defined in terms of Z t and the previous ones). 14 settembre / 31

63 AutoRegressive processes AR(1) [AutoRegressive] {X t } t Z is AR(1) if is stationary and X t = φx t 1 + Z t, t Z where φ R, {Z t } WN(0, σ 2 ). (1) (1) is an (infinite set of) equation. It is not obvious that a stationary process exists satisfying them (this will be discussed later). We are not saying {X t } t N is the Markov chain defined through X t = φx t 1 + Z t, t > 0 with X 0 some prescribed r.v. Now, assume a stationary process {X t } t Z exists satisfying (1) and E(X t Z s ) = 0 for t < s (this latter property seems natural as X t should be defined in terms of Z t and the previous ones). Then γ(0) = V(X t ) = E((φX t 1 + Z t ) 2 ) = φ 2 V(X t 1 ) + σ 2 + 2φE(X t 1 Z t ) = φ 2 γ(0) + σ 2. Hence γ(0) = σ2 1 φ 2 (makes sense only if φ 2 < 1 ). 14 settembre / 31

64 AutoRegressive processes. 2 Remarks: we have found φ 2 < 1 φ < 1 as a necessary condition for an AR(1) satisfying E(X t Z s ) = 0 for t < s. It will also be sufficient. Implicit assumption in the computations: E(X t ) = 0 (this can be proved analogously). More simply, one can then compute γ(h) for h > 0 (left for exercise). 14 settembre / 31