Basics: Definitions and Notation. Stationarity. A More Formal Definition

Transcription

1 Basics: Definitions and Notation A Univariate is a sequence of measurements of the same variable collected over (usually regular intervals of) time. Usual assumption in many time series techniques is that the time series data are stationary. Can define precisely, but taken for now as flat looking series. Zero trend, constant variance & constant autocorrelation structure over time. Variance: informally measures how far a set of (random) numbers are spread out from their mean Covariance: measures how one variable changes w.r.t. another. Correlation: scales the covariance from [ 1, 1]. Autocorrelation: signal correlation with itself at previous t Autocorrelation Function (ACF): signal power plot at previous t 71 / 287 Stationarity A More Formal Definition A stationary time series is one whose properties do not depend on the time at which the series is observed. So time series with trends/ seasonality, are not stationary Trend/ seasonality affect time series values at different t. As will be seen below a white noise 6 series is stationary. Stationary time series have no long-term predictable patterns. Plots show the series to be roughly horizontal (with possible cyclicity) with constant variance. 6 One where little/no autocorrelation outside certain limits 72 / 287

2 Stationary or Non-Stationary? (1/2) FIGURE 2.14 : Various Diverse 73 / 287 Stationary or Non-Stationary? (2/2) Looking at Fig Which are stationary (& why)? (d), (h) and (i) are ruled out due to Seasonality. similarly for (a), (c), (e), (f) & (i) due to Trend. (i) isn t a contender due to increasing Variance. just remains (b) and (g) as stationary series. Strong cycles in series (g) seem to make it non-stationary. But these cycles are aperiodic due to lynx population getting too big for available feed, so breeding stops and the population falls very low, then regenerating food sources lets numbers increase etc. Long-term, the timing of these cycles is not predictable. Hence the series is stationary. 74 / 287

3 Non-Stationary (1/5) From Fig again, see how DJ data was non-stationary in (a), but daily changes were stationary in (b). 1 Shows one way to make a series stationary difference consecutive values Differencing: Given Z t, make a new series Y t = Z t Z t 1 Y t will contain one less point than the original data. Can difference the data more than once, but once usually ok. 2 If there is a trend in the data, can fit a curve to the data and then model the residuals from that fit. fitting just needs to remove a long term trend, so use a line. 75 / 287 Non-Stationary (2/5): Example 4.1 Above methods should give series with constant location/ scale Although seasonality also violates stationarity, this is usually explicitly incorporated into the time series model. FIGURE 2.15 : A Rising Trend That can be Removed With a Line 76 / 287

4 Non-Stationary (3/5): Example 4.1 The data now have a constant location and variance, However, pattern of residuals shows them departing from the model in a systematic way, so not stationary. FIGURE 2.16 : Figure 2.15 With Trend Removed 77 / 287 Independent & Identically Distributed Data (IID) Univariate differ from standard linear regression models values in that data not always independent nor identically distributed. Sequence is IID if each random variable has same probability distribution and all are mutually independent. Mutually independent variables have a covariance of zero. Defining feature of such series is that ordering of list of values matters. The reason for this is that there is dependency and changing the order of the data could change its meaning. IID is important in the Central Limit Theorem, which states that probability distribution of the sum of IID variables with finite variance approaches a normal distribution. 78 / 287

5 Model Types Two basic types of time domain models: 1 These relate values now to past values & past prediction errors: A.k.a. Autoregressive Integrated Moving Average (ARIMA) models Want to understand & predict future values in this series. 3 parts in model: autoregressive (AR) & moving average (MA): AR part involves regressing variable on its own past values. MA part involves modelling the error as a linear combination of current error terms & at various times in the past Integrated as it can involve differencing the series We will look at these in more detail. 2 Ordinary regression models using time indices as x-variables. We have seen these previously in a previous lecture. Handy basic model & the basis of simple forecasting methods. Before choosing a model, look at above issues (trend, seasonality, stationarity etc.) 79 / 287 Non-Stationary (4/5): Example 4.1 again Differencing made DJ data in Fig (a) stationary ((b)). Need to look at auto-correlation function (ACF) of (a), (b). Shows that, (b) is more a candidate for stationarity than (a). FIGURE 2.17 : Figure 2.14 (a) DJ Index & (b) Differenced Data 80 / 287

6 Non-Stationary (5/5): Example 4.1 ACF ACF plot also can identify non-stationary time series. For stationary case, ACF drops to zero fast, non-stationary drops slowly. FIGURE 2.18 : Auto-Correlation Function of Figure 2.17 (a) & (b) Differenced DJ index (ACF) looks like a white noise series. These show just 1 autocorrelation outside 95%/ limits 2/ T Maybe DJ change is just random, uncorrelated with prior days. 81 / 287 Auto-Regressive Models The First-order Auto-Regressive Model (AR(1)) First examine the theoretical properties of AR(1) model. 1st order auto-regressive model is denoted as AR(1). In it, x at time t is a linear function of x at t 1 Algebraically, can express the model as follows: x t = c + φ 1 x t 1 + ɛ t (2.2) where c is constant, x t, x t 1 are the values of x at t, t 1 φ 1 is a parameter ɛ t is a random variable representing the white noise part Assumptions of the First-order Auto-Regressive Model: ɛ t is independently & normally distributed (mean 0, constant variance) & are independent of each other The time series x 1, x 2... is weakly stationary (implied by φ 1 < 1 ) 82 / 287

7 Auto-Regressive Models: Example 4.2 Earthquakes FIGURE 2.19 : Global Number of Earthquakes Magnitude 7.0 For 99 Years No firm trend in Fig 2.19: quakes = 20.2 shows series mean. Plot drifts either side of mean, remains briefly on each. There is no seasonality (annual data) and no obvious outliers. Hard to judge the variance. 83 / 287 Example 4.2: Original and Lag(1) Series t x t x t 1 (lag 1) TABLE 2.6 : Quakes V Lag(1) FIGURE 2.20 : Quakes V Lag(1) Table 2.6 shows first 5 values with their lag 1 values Fig 2.20: Quake dataset x t V x t 1 Only moderately related but there is a positive linear association so AR(1) model might be a useful model. 84 / 287

8 Reflections on Autocorrelation & ACF The autocorrelation function (ACF) of a series x t gives correlations between x t and lagged values of the series for lags of 1, 2, 3, and so on. Lagged values are x t 1, x t 2, x t 3, and so on. The ACF gives correlations between x t and x t 1, x t 2, x t 3 ACF is handy for identifying the possible structure of time series. Can be tricky as often isn t a single clear interpretation of a sample ACF. The ACF of the residuals for a model is also useful. The ideal for a sample ACF of residuals is that there aren t any significant correlations for any lag. 85 / 287 Reflections on Autocorrelation & ACF: PACF Given 3 points x 1, x 2, x 3, ACF finds how e.g. x 1, x 2 are correlated The value of correlation obtained is technically not true value of correlation, because the x 2 value will be inspired by x 3 value. Partial Auto-Correlation Function (PACF) essentially strips out x 3 part. So PACF is that part of the correlation between x 1 and x 2, not explained by the correlation of x 3 in x 2. Example of this shown in Fig 2.21 Pink & Green boxes show how x 1 & x 2 correlated, Correlation between x 2 & x 3 shown in Green. So partial correlation of x 1, x 2 = correlation that of x 2 & x 3 (Green box), shown by Pink box. FIGURE 2.21 : PACFs 86 / 287

9 Example 4.2: Residuals & ACF in an AR(1) Model FIGURE 2.22 : Residual and ACF of Fig Fitting the data in Fig 2.20 to the AR(1) model Eqn.(2.2) gives the following relationship: Quakes, x t = x t 1 Residuals plot Fig 2.22(L) doesn t show any serious problems (maybe an outlier at a fitted value of about 28?). Auto-Correlation function (R) seems ok too; all are within 95% lines. 87 / 287 Auto-Regressive Models: ACF Pattern of ACF for AR(1) Model The sign of φ 1 thrown up dictates the pattern of the ACF plot: For +φ 1, ACF exponentially 0 as lag increases. For φ 1, ACF also exponentially 0 as lag increases, but the algebraic signs for the autocorrelations alternate between + & Fig 2.23 shows ACFs of an AR(1) with (a) φ 1 = 0.6 (Note how it tapers), (b) φ 1 = 0.7 (Note the oscillatory pattern) (a) φ 1 = 0.6 (b) φ 1 = 0.7 FIGURE 2.23 : Two ACF Plots for Series Wth Different φ 1 88 / 287

10 AR Models: Example 4.3 LA Deaths (1/2) FIGURE 2.24 : Deaths due to CV Diseases in LA County Slight downward trend in Fig 2.24 so the series may not be stationary. To render it (possibly) stationary, take first differences y t = x t x t 1. Fit with line: constant differences in average y for each unit x change 89 / 287 AR Models: Example 4.3 LA Deaths (2/2) Fig 2.25 (a) looks like an AR(1) with a negative lag 1 autocorrelation. Note from (b) that the lag 2 correlation is roughly equal to the squared value of the lag 1 correlation. (a) First Differences (b) ACF of First Differences FIGURE 2.25 : Time Plot & ACF of First Differences for Fig / 287

11 A Small Change in Terminology Types of Model seen so far: With autoregressive terms is called an AR model. When only moving average terms involved, called MA model. With both MA & AR terms, & no differencing, is known as ARMA. Most s/w model attributes have form (AR order, differencing, MA order) ARIMA models can have AR & MA terms, and differencing operations: A model with (only) two AR terms is an ARIMA of order (2,0,0) A MA(2) model would be specified as an ARIMA of order (0,0,2) ARIMA(1,1,1) model has 1 AR term, 1 MA term & first differencing of the trend component. Can also have a ARIMA(1,2,1), with first difference of first differences This type of difference could account for a quadratic trend in the data. 91 / 287 So-Called Moving Average models in ARMA Models Rather than use past x t in a regression, this model uses past forecast errors in a regression-like model. Unfortunately commonly referred to as Moving Average or MA model NB: Don t mix up MA models & MA smoothing earlier: former to forecast, latter to estimate trends. Moving Average models take the following form: x t = µ + ɛ t + θ 1 ɛ t θ i ɛ t i = µ + ɛ t + Σ q i=1 θ iɛ t i (2.3) where µ is the mean and is usually zero θ i are the parameters, q is the order (i.e. (MA(q)) ɛ t, ɛ t i etc: random (independent) white noise variable parts Of course, can t see values of ɛ t, not really usual regression. Notice can think of x t as a weighted MA of past few forecast errors. Different θ i parameters give different time series patterns. Whereas, as per AR models, variance of ɛ t only alters series scale. 92 / 287

12 Moving Average Models: Example 4.4 Fig 2.26 (a), (b) are plots from MA models with q different parameters. There are differences in resolution apparent with (b) more fine-grained In both, θ t is normally distributed white noise with mean 0, variance 1. (a) x t = 20 + ɛ t + 0.8ɛ t 1 (b) x t = 20 + ɛ t ɛ t ɛ t 2 (a) MA(1) Model (b) MA(2) Model FIGURE 2.26 : Some Moving Average (MA(q)) Models 93 / 287 Moving Average Models: Example 4.5 Time Plot of x t = 10 + ɛ t + 0.7ɛ t 1 (ɛ t ~iid N(0,1)) shown in Fig 2.27(a). Not possible to tell much from this () plot. ACF in Fig 2.27(b) shows a spike at lag 1 (with insignificant lags past 1 after) showing that it is a MA(1) series. (a) Plot (b) ACF FIGURE 2.27 : & ACF of a Moving Average (MA(1)) Model 94 / 287

13 Moving Average Models: Example 4.6 Fig 2.28(a) shows x t = 10 + ɛ t + 0.7ɛ t ɛ t 2 plot (ɛ t ~iid N(0,1)). As per Fig 2.27(a) not possible to tell much from it. ACF in Fig 2.28(b) shows two statistically significant spikes at lags 1 and 2 followed by non-significant values for other lags. (a) Plot (b) ACF FIGURE 2.28 : & ACF of a Moving Average (MA(2)) Model 95 / 287 Knowing Your AR s From Your MA s Some Useful Facts About PACF and ACF Patterns: Identification of an AR model is often best done with the PACF. For AR models, theoretical PACF shuts off past the model order. This means theory the PACFs are 0 beyond that point. Or number of non-zero partial autocorrelations gives the AR model order To identify MA models best do an ACF rather than PACF. For MA case, theoretical PACF doesn t shut off, but goes to 0 in some manner. MA model has a clearer pattern in the ACF. The ACF will have non-zero autocorrelations only at lags involved in the model. 96 / 287

14 ACF & PACF: Example 4.2 Earthquakes again... Recall Fig 2.19 The Earthquakes series ACF & PACF shown in Fig 2.30(a), (b) Observe how the latter drops to zero quicker FIGURE 2.29 : Quakes (a) Earthquakes ACF (b) Earthquakes PACF FIGURE 2.30 : ACF & PACF of Fig / 287 ACF & PACF: Back to Example 4.5 Recall the MA(1) Time Plot of x t = 10 + ɛ t + 0.7ɛ t 1 in Fig 2.27(a). This is shown again in Fig. 2.31(a) ACF in Fig 2.31(b) tapers to 0 faster than PACF in Fig 2.31 (c). (a) Plot (b) ACF (c) PACF FIGURE 2.31 : & ACF of a Moving Average (MA(1)) Model 98 / 287

15 A Summary Slide FIGURE 2.32 : When to Use The Different Models 99 / 287 Summary In this section, covered many basic aspects of : Outlining the main objectives of time series analysis Inferring key aspects arising out of studying time series Describing the trend in the series using Moving Averages & Linear Regression Estimating the Key Patterns (Trend/ Seasonal/ Cyclical effects) in a series & removing them. Identifying & interpreting future values using first-order autoregressive & moving average models. 100 / 287