STAT 153: Introduction to Time Series Instructor: Aditya Guntuboyina Lectures: 12:30 pm - 2 pm (Tuesdays and Thursdays) Office Hours: 10 am - 11 am (Tuesdays and Thursdays) 423 Evans Hall GSI: Brianna Heggeseth Section: 10 am - 12 pm or 12 pm - 2 pm (Fridays) Office Hours and Location: TBA Announcements, Lecture slides, Assignments, etc. will be posted on the course site at bspace.
Course Outline A Time Series is a set of numerical observations, each one being recorded at a specific time. Examples of Time Series data are ubiquitous. The aim of this course is to teach you how to analyze such data.
Population of the United States US Population 0.0e+00 1.0e+08 2.0e+08 3.0e+08 1800 1850 1900 1950 2000 Year(once every ten years)
Course Outline (continued) There are two approaches to time series analysis: Time Domain Approach Frequency Domain Approach (also known as the Spectral or Fourier analysis of time series) Very roughly, 60% of the course will be on Time Domain methods and 40% on Frequency Domain methods.
Time Domain Approach Seeks an answer to the following question: Given the observed time series, how does one guess future values? Forecasting or Prediction
Time Domain (continued) Forecasting is carried out through three steps: Find a MODEL that adequately describes the evolution of the observed time series over time. Fit this model to the data. In other words, estimate the parameters of the model. Forecast based on the fitted model.
Time Series Models Most of our focus in the Time Domain part of the course will be on the following two classes of models: Trend + Seasonality + Stationary ARMA Differencing + Stationary ARMA (consist of the ARIMA and Seasonal ARIMA models) In the Time Domain Part of the course, we study these models and learn how to execute each of the three steps outlined in the previous slide with them.
Time Series Models (continued) These provide a sturdy toolkit for analyzing many practical time series data sets. State-Space models are a modern and very powerful class of time series models. Forecasting in these models is carried out via an algorithm known as the Kalman Filter. We shall spend some time on these models although we will not have time to study them in depth.
Frequency Domain Approach Brightness of a variable star on 600 consecutive nights Brightness 0 5 10 15 20 25 30 35 0 100 200 300 400 500 600 day
Frequency Domain (continued) Based on the idea that the observed time series is made up of CYCLES having different frequencies. In the Frequency Domain Approach, the data is analyzed by discovering these hidden cycles along with their relative strengths. The key tool here is the Discrete Fourier Transform (DFT) or, more specifically, a function of the DFT known as the Periodogram.
Frequency Domain (continued) In the Frequency Domain part of the course, we shall study the periodogram and its performance in discovering periodicities when the data are indeed made up of many different cycles. It turns out that the raw periodogram is often too variable as an estimator of the true Spectrum and we shall study methods for improving it.
Rest of this Lecture Some more Time Series Data Examples Simplest Time Series Model: Purely Random Process (Section 3.4.1) Sample Autocorrelation Coefficients and the Correlogram (Section 2.7 and Page 56)
Annual Measurements of the level of Lake Huron 1875-1972 Level in feet 576 577 578 579 580 581 582 1880 1900 1920 1940 1960 Year
Monthly Accidental Deaths in the US from 1973-1978 Number of Deaths 7000 8000 9000 10000 11000 1973 1974 1975 1976 1977 1978 1979 Time
The first step in the time domain analysis of a time series data set is to find a model that well describes the evolution of the data over time. Basic Modelling Strategy: Start Simple Build Up Simplest Model: X,t = 1,,n independent N ( 0,σ 2 ) t Purely Random Process or Gaussian White Noise
100 Observations from Gaussian White Noise with unit Variance Purely Random Process -2-1 0 1 2 3 0 20 40 60 80 100 Time
Is this data set from a purely random process Data -3-2 -1 0 1 2 0 20 40 60 80 100 Time
How to check if a given time series is purely random?
How to check if a given time series is purely random? Answer: Think in terms of Forecasting.
How to check if a given time series is purely random? Answer: Think in terms of Forecasting. For a purely random series, the given data can NOT help in predicting Xn+1. The best estimate of Xn+1 is E(Xn+1) = 0. In particular, X1 can not predict X2 and X2 can not predict X3 and so on.
How to check if a given time series is purely random? Answer: Think in terms of Forecasting. For a purely random series, the given data can NOT help in predicting Xn+1. The best estimate of Xn+1 is E(Xn+1) = 0. In particular, X1 can not predict X2 and X2 can not predict X3 and so on. Therefore, the correlation coefficient between Y = (X1,..., Xn-1) and Z = (X2,..., Xn) must be close to zero.
The formula for the correlation between Y and Z is r = n 1 t =1 n 1 t =1 ( X t X )( (1) X t +1 X ) (2) n 1 t =1 ( X t X ) 2 (1) X t +1 X (2) ( ) 2 X (1) = n 1 X t t =1 n 1 X (2) = n 1 X t +1 t =1 n 1 This formula is usually simplified to obtain r 1 = n 1 t =1 ( X t X) ( X t +1 X) n t =1 ( X t X) 2 X = n X t t =1 n Note the subscript on the left hand side above.
Sample Autocorrelation Coefficients The quantity r1 is called the Sample Autocorrelation Coefficient of X1,..., Xn at lag one. Lag one because this correlation is between Xt and Xt+1. When X1,..., Xn are obtained from a Purely Random Process, r1 is close to zero, particularly when n is large. One can similarly consider Sample Autocorrelations at other lags: r k = n k t =1 ( X t X) ( X t + k X) n t =1 ( X t X) 2 k = 1, 2,...
Correlogram Mathematical Fact: When X1,..., Xn are obtained from a Purely Random process, r1, r2,... are independently distributed according to N(0, 1/n). So one way of testing if the series is purely random is to plot the sample autocorrelations. This plot is known as the Correlogram. Use the function acf() in R to get the Correlogram. ts.obs = rnorm(100) acf(ts.obs, lag.max = 20, type = correlation, plot = T, drop.lag.0 = F)
Correlogram of a Purely Random Series of 100 Observations ACF -0.2 0.0 0.2 0.4 0.6 0.8 1.0 0 5 10 15 20 Lag The correlogram plots rk against k. r0 always equals 1. The blue bands correspond to levels of ±1.96/ n
Interpreting the Correlogram When X1,..., Xn are obtained from a Purely Random process, the probability that a fixed rk lies outside the blue bands equals 0.05. A value of rk outside the blue bands is significant i.e., it gives evidence against pure randomness. However, the overall probability of getting at least one rk outside the bands increases with the number of coefficients plotted. If 20 rks are plotted, one expects to get one significant value under pure randomness.
Rules of Thumb (for deciding if a correlogram indicates departure from randomness) Chatfield (page 56) A single rk just outside the bands may be ignored, but two or three values well outside indicate a departure from pure randomness. A single significant rk at a lag which has some physical interpretation such as lag one or a lag corresponding to seasonal variation also indicates evidence of non-randomness.
Is this data set from a purely random process Data -3-2 -1 0 1 2 0 20 40 60 80 100 Time
Correlogram of the Data in the Previous Slide ACF -0.2 0.0 0.2 0.4 0.6 0.8 1.0 0 5 10 15 20 Lag This data was generated from a moving average process.
Is this data set from a purely random process? Data -3-2 -1 0 1 2 3 0 20 40 60 80 100 Time
Correlogram of the Data in the Previous Slide ACF -0.4-0.2 0.0 0.2 0.4 0.6 0.8 1.0 0 5 10 15 20 Lag Again, there is more structure in this dataset compared to pure randomness.
Is this data set from a purely random process? Data -3-2 -1 0 1 2 0 20 40 60 80 100 Time
Correlogram of the Data in the Previous Slide ACF -0.2 0.0 0.2 0.4 0.6 0.8 1.0 0 5 10 15 20 Lag Lots of structure here.
Conclusions for Today Purely Random Process or Gaussian White Noise. Sample Autocorrelation Coefficient, rk. Correlogram. How do rks behave under pure randomness? How to tell from the Correlogram if there is evidence of departure from pure randomness?