Module 3. Descriptive Time Series Statistics and Introduction to Time Series Models

Transcription

1 Module 3 Descriptive Time Series Statistics and Introduction to Time Series Models Class notes for Statistics 451: Applied Time Series Iowa State University Copyright 2015 W Q Meeker November 11, h 20min 3-1

2 Module 3 Segment 1 Populations, Processes, and Descriptive Time Series Statistics 3-2

3 Populations A population is a collection of identifiable units (or a specified characteristic of these units) A frame is a listing of the units in the population We take a sample from the frame and use the resulting data to make inferences about the population Simple random sampling with replacement from a population implies independent and identically distributed (iid) observations Standard statistical methods use the iid assumption for observations or residuals (in regression) 3-3

4 Processes A process is a system that transforms inputs into outputs, operating over time inputs outputs A process generates a sequence of observations (data) over time We can use a realization from the process to make inferences about the process The iid model is rarely appropriate for processes (observations close together in time are typically correlated and the process often changes with time) 3-4

5 Process, Realization, Model, Inference, and Applications Description Process Generates Data Realization Z, Z,, Z 1 2 n Statistical Model for the Process (Stochastic Process Model) Estimates for Model Parameters Inferences Forecasts Control 3-5

6 Stationary Stochastic Processes Z t is a stochastic process Some properties of Z t include mean µ t, variance σ 2 t, and autocorrelation ρ Z t,z t+k In general, these can change over time Strongly stationary (also strictly or completely stationary): Difficult to check F(z t1,,z tn ) = F(z t1 +k,,z tn +k) Weakly stationary (or 2nd order or covariance stationary) requires only that µ = µ t and σ 2 = σ 2 t be constant and that ρ k = ρ Zt,Z t+k depend only on k Easy to check with sample statistics Generally, stationary is understood to be weakly stationary 3-6

7 Change in Business Inventories Stationary? Billions of Dollars Year 3-7

8 Estimation of Stationary Process Parameters Some Notation Process Parameter Notation Estimate Mean of Z µ Z = E(Z) µ Z = z = nt=1 z t n Variance of Z γ 0 = σ 2 Z = Var(Z) γ 0 = nt=1 (z t z) 2 n Standard σ Z σ Z = γ 0 Deviation Autocovariance γ k γ k = n k t=1 (z t z)(z t+k z) n Autocorrelation ρ k = γ k γ 0 ρ k = γ k γ 0 Variance of Z σ 2 Z = Var( Z) S 2 Z = γ 0 n [ ] 3-8

9 Change in Business Inventories with z and z ±3 σ Z Billions of Dollars Year 3-9

10 Variance of Z with Correlated Data The Variance of Z σ 2 Z = Var( Z) = γ 0 n n 1 k= (n 1) ( 1 k n with correlated data is more complicated than with iid data where σ 2 Z = Var( Z) = γ 0 n = σ2 n Thus if one incorrectly uses the iid formula, the variance is under estimated, potentially leading for false conclusions ) ρ k 3-10

11 Module 3 Segment 2 Correlation and Autocorrelation 3-11

12 Correlation [From Statistics 101 ] Consider random data y and x (eg, sales and advertising): x y x 1 y 1 x 2 y 2 x n y n ρ x,y denotes the population correlation between all values of x and y in the population To estimate ρ x,y, we use the sample correlation ρ x,y = ni=1 (x i x)(y i ȳ) ni=1 (x i x) 2 (Stat 101) n i=1 (y i ȳ) 2, 1 ρ x,y

13 Sample Autocorrelation Consider the random time series realization z 1,z 2,,z n Lagged Variables t z t z t+1 z t+2 z t+3 z t+4 1 z 1 z 2 z 3 z 4 z 5 2 z 2 z 3 z 4 z 5 z 6 3 z 3 z 4 z 5 z 6 z 7 n 2 z n 2 z n 1 z n n 1 z n 1 z n n z n Assuming weak (covariance) stationarity, let ρ k denote the process correlation between observations separated by k time periods To compute the order k sample autocorrelation (ie, correlation between z t and z t+k ) ρ k = γ k γ 0 = Note: 1 ρ k 1 n k t=1 (z t z)(z t+k z) nt=1 (z t z) 2, k = 0,1,2, 3-13

14 Sample Autocorrelation (alternative formula) Consider the random time series realization z 1,z 2,,z n Lagged Variables t z t z t 1 z t 2 z t 3 z t 4 1 z 1 2 z 2 z 1 3 z 3 z 2 z 1 4 z 4 z 3 z 2 z 1 n z n z n 1 z n 2 z n 3 z n 4 Assuming weak (covariance) stationarity, let ρ k denote the process correlation between observations separated by k time periods To compute the order k sample autocorrelation (ie, correlation between z t and z t k ) ρ k = γ nt=k+1 k (z t z)(z t k z) = γ nt=1 0 (z t z) 2, k = 0,1,2, Note: 1 ρ k

15 Wolfer Sunspot Numbers

16 Lagged Sunspot Data Lagged Variables t Spot Spot1 Spot2 Spot3 Spot

17 Wolfer Sunspot Numbers Correlation Between Observations Separated by One Time Period spotts[-1] spotts[-100] Correlation = Autocorrelation =

18 Wolfer Sunspot Numbers Correlation Between Observations Separated by k Time Periods [showacf(spotts)] timeseries timeseries Lag 1 series Series lag 1 Lag 2 series Series lag 2 Lag 3 series Series lag 3 Lag 4 series Series lag 4 Lag 5 series Series lag 5 Lag 6 series Series lag 6 Lag 7 series Series lag 7 Lag 8 series Series lag 8 Lag 9 series Series lag Lag 10 series Series lag Lag 11 series Series lag Lag 12 series Series lag Lag 13 series Series lag Lag ACF Series : timeseries 3-18

19 Wolfer Sunspot Numbers Sample ACF Function Series : spotts ACF Lag 3-19

20 Module 3 Segment 3 Autoregression, Autoregressive Modeling, and and Introduction to Partial Autocorrelation 3-20

21 Wolfer Sunspot Numbers

22 Autoregressive (AR) Models AR(0): z t = µ+a t (White noise or Trivial model) AR(1): z t = θ 0 +φ 1 z t 1 +a t AR(2): z t = θ 0 +φ 1 z t 1 +φ 2 z t 2 +a t AR(3): z t = θ 0 +φ 1 z t 1 +φ 2 z t 2 +φ 3 z t 3 +a t AR(p): z t = θ 0 +φ 1 z t 1 +φ 2 z t 2 + +φ p z t p +a t a t nid(0,σa 2) 3-22

23 Data Analysis Strategy Tentative Identification Time Series Plot Range-Mean Plot ACF and PACF Estimation Least Squares or Maximum Likelihood Diagnostic Checking Residual Analysis and Forecasts No Model ok? Yes Use the Model Forecasting Explanation Control 3-23

24 AR(1) Model for the Wolfer Sunspot Data Data and 1-Step Ahead Predictions Sunspot Residuals, AR(1) Model Year Number Year Number Series : residuals(spotar1fit) ACF residuals(spotar1fit) Lag Quantiles of Standard Normal 3-24

25 AR(2) Model for the Wolfer Sunspot Data Data and 1-Step Ahead Predictions Sunspot Residuals, AR(2) Model Year Number Year Number Series : residuals(spotar2fit) ACF residuals(spotar2fit) Lag Quantiles of Standard Normal 3-25

26 AR(3) Model for the Wolfer Sunspot Data Data and 1-Step Ahead Predictions Sunspot Residuals, AR(3) Model Year Number Year Number Series : residuals(spotar3fit) ACF residuals(spotar2fit) Lag Quantiles of Standard Normal 3-26

27 AR(4) Model for the Wolfer Sunspot Data Data and 1-Step Ahead Predictions Sunspot Residuals, AR(4) Model Year Number Year Number Series : residuals(spotar4fit) ACF residuals(spotar2fit) Lag Quantiles of Standard Normal 3-27

28 Sample Partial Autocorrelation Function The sample PACF φ kk can be computed from the sample ACF ρ 1, ρ 2, using the recursive formula from Durbin (1960): φ 1,1 = ρ 1 where φ kk = ρ k 1 k 1 j=1 k 1 j=1 φ k 1, j ρ k j φ k 1, j ρ j, k = 2,3, φ kj = φ k 1,j φ kk φ k 1,k j (k = 3,4,; j = 1,2,,k 1) Thus the PACF contains no new information just a different way of looking at the information in the ACF This sample PACF formula is based on the solution of the Yule-Walker equations (covered module 5) 3-28

29 Summary of Sunspot Autoregressions Order Regression Output PACF Model p R 2 S φ p t φp φ pp AR(1) AR(2) AR(3) AR(4) φ p is from ordinary least squares (OLS) φ pp is from the Durbin formula 3-29

30 Sample Partial Autocorrelation The true partial autocorrelation function, denoted by φ kk, for k = 1,2, is the process correlation between observations separated by k time periods (ie, between z t and z t+k ) with the effect of the intermediate z t+1,,z t+k 1 removed We can estimate φ kk, k = 1,2, with φ kk, k = 1,2, φ 1,1 = φ 1 from the AR(1) model φ 2,2 = φ 2 from the AR(2) model φ kk = φ k from the AR(k) model The Durbin formula gives somewhat different answers due to the basis of the estimator (ordinary least squares versus solution of the Yule-Walker equations) 3-30

31 Module 3 Segment 4 Introduction to ARMA Models and Time Series Modeling 3-31

32 Autoregressive-Moving Average (ARMA) Models (aka Box-Jenkins Models) AR(0): z t = µ+a t (White noise or Trivial model) AR(1): z t = θ 0 +φ 1 z t 1 +a t AR(2): z t = θ 0 +φ 1 z t 1 +φ 2 z t 2 +a t AR(p): z t = θ 0 +φ 1 z t 1 + +φ p z t p +a t MA(1): z t = θ 0 θ 1 a t 1 +a t MA(2): z t = θ 0 θ 1 a t 1 θ 2 a t 2 +a t MA(q): z t = θ 0 θ 1 a t 1 θ q a t q +a t ARMA(1,1): z t = θ 0 +φ 1 z t 1 θ 1 a t 1 +a t ARMA(p,q): z t = θ 0 +φ 1 z t 1 + +φ p z t p θ 1 a t 1 θ q a t q +a t a t nid(0,σ 2 a) 3-32

33 Graphical Output from RTSERIES Function iden(spottsd) Wolfer Sunspots w= Number of Spots Range-Mean Plot w time ACF Range ACF Partial ACF Lag PACF Mean Lag 3-33

34 Tabular Output from RTSERIES Function iden(spottsd) Identification Output for Wolfer Sunspots w= Number of Spots [1] "Standard deviation of the working series= " ACF Lag ACF se t-ratio Partial ACF Lag Partial ACF se t-ratio

35 Standard Errors for Sample Autocorrelations and Sample Partial Autocorrelations Sample ACF standard error: S ρk = ( 1+2 ρ ρ 2 ) k 1 n Also can compute the t-like statistics t = ρ k /S ρk Sample PACF standard error: S φkk = 1/ n Use to compute t-like statistics t = φ kk /S φkk and set decision bounds In long realizations from a stationary process, if the true parameter is really 0, then the distribution of a t-like statistic can be approximated by N(0, 1) Values of ρ k and φ kk may be judged to be different from zero if the t-like statistics are outside specified limits (±2 is often suggested; might use ±15 as a warning ) 3-35

36 Drawing Conclusions from Sample ACF s and PACF s PACF PACF Dies Down Cutoff after p > 1 ACF Dies Down ARMA AR(p) ACF Cutoff after q > 1 M A(q) Impossible Technical details, background, and numerous examples will be presented in Modules 4 and

37 Comments on Drawing Conclusions from Sample ACF s and PACF s The t-like statistics should only be used as guidelines in model building because: An ARMA model is only an approximation to some true process Sampling distributions are complicated; the t-like statistics do not really follow a standard normal distribution (only approximately, in large samples, with a stationary process) Correlations among the ρ k values for different k (eg, ρ 1 and ρ 2 may be correlated) Problems relating to simultaneous inference (looking at many different statistics simultaneously, confidence/significance levels have little meaning) 3-37