Spectral Analysis Al Nosedal University of Toronto Winter 2016 Al Nosedal University of Toronto Spectral Analysis Winter 2016 1 / 71
essentially, all models are wrong, but some are useful George E. P. Box (one of the great statistical minds of the 20th century). Al Nosedal University of Toronto Spectral Analysis Winter 2016 2 / 71
Description of data The data in sunspots shows yearly numbers of sunspots from 1771 to 1870. Al Nosedal University of Toronto Spectral Analysis Winter 2016 3 / 71
Reading data sunspots<-read.table(file="sunspots.dat",header=false); sunspots<-ts(sunspots,start=1771); Al Nosedal University of Toronto Spectral Analysis Winter 2016 4 / 71
Exercise 1. Make the time series plot of the sunspots.dat. 2. Make the correlogram (ACF) of the series. Al Nosedal University of Toronto Spectral Analysis Winter 2016 5 / 71
Time Series Plot sunspot activity 0 50 150 1780 1800 1820 1840 1860 Time Al Nosedal University of Toronto Spectral Analysis Winter 2016 6 / 71
Correlogram of 11-year differences for sunspot data V1 ACF 0.2 0.4 1.0 0 5 10 15 20 Lag Al Nosedal University of Toronto Spectral Analysis Winter 2016 7 / 71
Before attempting to model the series, the cyclical component needs to be removed. A simple procedure which is useful here is to take appropriate differences of the series. Here, assuming an 11 year cycle, differences between points 11 years apart are used Y t = X t X t 11 (The resulting series should contain no obvious periodic component). Al Nosedal University of Toronto Spectral Analysis Winter 2016 8 / 71
Exercise 1. Find Y t = X t X t 11 using R. 2. Make the time series plot of Y t. 3. Make the correlogram (ACF) of Y t. 4. Fit AR model using ar.yw. 5. Plot residuals from your model. Al Nosedal University of Toronto Spectral Analysis Winter 2016 9 / 71
New Series (sunspots) newsun<-sunspots[12:100] - sunspots[1:89] Al Nosedal University of Toronto Spectral Analysis Winter 2016 10 / 71
Time Series Plot (newsun) sunspot activity 100 0 100 0 20 40 60 80 Time Al Nosedal University of Toronto Spectral Analysis Winter 2016 11 / 71
Correlogram Series newsun ACF 0.2 0.4 1.0 0 5 10 15 Lag Al Nosedal University of Toronto Spectral Analysis Winter 2016 12 / 71
Fitting Model sun.ar<-ar.yw(newsun); sun.ar ## ## Call: ## ar.yw.default(x = newsun) ## ## Coefficients: ## 1 2 3 4 5 ## 1.5009-1.1516 0.6555-0.4361 0.2092 ## ## Order selected 5 sigma^2 estimated as 311.5 Al Nosedal University of Toronto Spectral Analysis Winter 2016 13 / 71
Plot of Akaike s criterion plot.ts(sun.ar$aic, xlab="order", ylab="akaike's criterion"); Al Nosedal University of Toronto Spectral Analysis Winter 2016 14 / 71
Plot of Akaike s criterion Akaike's criterion 0 40 100 5 10 15 20 Order Al Nosedal University of Toronto Spectral Analysis Winter 2016 15 / 71
Plot of Residuals plot.ts(sun.ar$resid, xlab="year", ylab="residual from AR Model"); abline(h=0, col="red"); Al Nosedal University of Toronto Spectral Analysis Winter 2016 16 / 71
Plot of Residuals Residual from AR Model 60 0 40 0 20 40 60 80 Year Al Nosedal University of Toronto Spectral Analysis Winter 2016 17 / 71
ACF of Residuals acf(sun.ar$resid); Al Nosedal University of Toronto Spectral Analysis Winter 2016 18 / 71
ACF of Residuals Series sun.ar$resid ACF 0.2 0.4 1.0 0 5 10 15 Lag Al Nosedal University of Toronto Spectral Analysis Winter 2016 19 / 71
All the correlations (apart from the one corresponding to lag 11) are small and lie within the horizontal bands, indicating that they do not differ significantly from zero. This suggests that the fitted autoregressive model is a reasonable fit for the data. Al Nosedal University of Toronto Spectral Analysis Winter 2016 20 / 71
Spectral Density If a time series {Y t } has autocovariance γ(k) satisfying k= γ(k) <, then we define its spectral density as f (w) = k= γ(k)e 2πiwk for < w <. We define its normalized spectral density as f (w) = k= ρ(k)e 2πiwk for < w < (where ρ(k) represents its autocorrelation function). Al Nosedal University of Toronto Spectral Analysis Winter 2016 21 / 71
Periodogram The periodogram is the sample estimate of the power spectrum (or spectral density). It is given by n 1 ˆf (w) = Ĉ(0) + 2 Ĉ(k)cos(2πkw). k=1 Al Nosedal University of Toronto Spectral Analysis Winter 2016 22 / 71
Why is the periodogram useful? The periodogram is ideal for identifying periodicity in data and estimating the frequency of the period. Consider a very simple periodic pattern with no noise producing the Figure shown below (see R code on next slide). Al Nosedal University of Toronto Spectral Analysis Winter 2016 23 / 71
n<-200; x<-c(1:n); y<-5+4*cos(2*pi*x/10 + 2.5); plot(x,y,type="l"); title("pure periodic series"); Al Nosedal University of Toronto Spectral Analysis Winter 2016 24 / 71
Pure periodic series y 2 4 6 8 0 50 100 150 200 x Al Nosedal University of Toronto Spectral Analysis Winter 2016 25 / 71
Periodogram (R code) z<-spec.pgram(y,fast=false, taper =0.0); Al Nosedal University of Toronto Spectral Analysis Winter 2016 26 / 71
Periodogram (Graph) Series: y Raw Periodogram spectrum 1e 05 1e+01 0.0 0.1 0.2 0.3 0.4 0.5 frequency bandwidth = 0.00144 Al Nosedal University of Toronto Spectral Analysis Winter 2016 27 / 71
Noisy version n<-200; x<-c(1:n); y<-5+4*cos(2*pi*x/10 + 2.5); err<-rnorm(n,0,0.25); y_err<-y + err; plot(x,y_err,type="l"); title("periodic series with noise"); Al Nosedal University of Toronto Spectral Analysis Winter 2016 28 / 71
Periodic series with noise y_err 2 4 6 8 0 50 100 150 200 x Al Nosedal University of Toronto Spectral Analysis Winter 2016 29 / 71
Periodogram (R code) z_err<-spec.pgram(y_err,fast=false, taper =0.0); Al Nosedal University of Toronto Spectral Analysis Winter 2016 30 / 71
Periodogram (Graph) Series: y_err Raw Periodogram spectrum 1e 03 1e+01 0.0 0.1 0.2 0.3 0.4 0.5 frequency bandwidth = 0.00144 Al Nosedal University of Toronto Spectral Analysis Winter 2016 31 / 71
The Sunspot Data (R Code) sunspots<-read.table(file="sunspots.dat",header=false); sunspots<-ts(sunspots,start=1771); z_sun<-spec.pgram(sunspots,fast=false, taper =0.0); Al Nosedal University of Toronto Spectral Analysis Winter 2016 32 / 71
The Sunspot Data (Graph) Series: sunspots Raw Periodogram spectrum 1 100 0.0 0.1 0.2 0.3 0.4 0.5 frequency bandwidth = 0.00289 Al Nosedal University of Toronto Spectral Analysis Winter 2016 33 / 71
The Sunspot Data (R Code...again) names(z_sun); plot(z_sun$freq,z_sun$spec,type="l"); Al Nosedal University of Toronto Spectral Analysis Winter 2016 34 / 71
The Sunspot Data (Graph) z_sun$spec 0 6000 14000 0.0 0.1 0.2 0.3 0.4 0.5 z_sun$freq Al Nosedal University of Toronto Spectral Analysis Winter 2016 35 / 71
The Sunspot Data (R Code...again) names(z_sun); plot(z_sun$freq,log(z_sun$spec),type="l"); Al Nosedal University of Toronto Spectral Analysis Winter 2016 36 / 71
The Sunspot Data (Graph) log(z_sun$spec) 0 4 8 0.0 0.1 0.2 0.3 0.4 0.5 z_sun$freq Al Nosedal University of Toronto Spectral Analysis Winter 2016 37 / 71
A General Approach to Time Series Modeling Plot the series and examine the main features of the graph, checking in particular whether there is a) a trend, b) a seasonal component, c) any apparent sharp changes in behavior, d) any outlying observations. Al Nosedal University of Toronto Spectral Analysis Winter 2016 38 / 71
A General Approach to Time Series Modeling Remove the trend and seasonal components to get stationary residuals. To achieve this goal it may sometimes be necessary to apply a preliminary transformation to the data. For example, if the magnitude of the fluctuations appears to grow roughly linearly with the level of the series, then the transformed series ln(x t ) will have fluctuations of more constant magnitude. There are several ways in which trend and seasonality can be removed, some involving estimating components and subtracting them from the data, and others depending on differencing the data, i.e., replacing the original series X t by Y t = X t X t d for some positive integer d. Whichever method is used, the aim is to produce a stationary series, whose values we shall refer to as residuals. Al Nosedal University of Toronto Spectral Analysis Winter 2016 39 / 71
A General Approach to Time Series Modeling Choose a model to fit the residuals. Forecasting will be achieved by forecasting the residuals and then inverting the transformations described above to arrive at forecasts of the original series X t. Al Nosedal University of Toronto Spectral Analysis Winter 2016 40 / 71
Example: Accidental deaths The file deaths.dat contains monthly accidental death figures in the U. S. A. from 1973 to 1978. Al Nosedal University of Toronto Spectral Analysis Winter 2016 41 / 71
Reading data deaths<-read.table(file="deaths.dat",header=false); deaths<-unlist(deaths); deaths.ts<-ts(deaths,start=1973, freq=12); Al Nosedal University of Toronto Spectral Analysis Winter 2016 42 / 71
Time Series Plot Accidental deaths thousands 7000 10000 1973 1975 1977 1979 Time Al Nosedal University of Toronto Spectral Analysis Winter 2016 43 / 71
This time series plot shows a strong seasonal pattern. We shall consider the problem of representing the data as the sum of a trend, a seasonal component, and a residual term. Al Nosedal University of Toronto Spectral Analysis Winter 2016 44 / 71
Periodogram (R Code) z_deaths<-spec.pgram(deaths.ts,fast=false, taper =0.0,log="no"); Al Nosedal University of Toronto Spectral Analysis Winter 2016 45 / 71
Periodogram (Graph) Series: deaths.ts Raw Periodogram spectrum 0 1000000 0 1 2 3 4 5 6 frequency bandwidth = 0.0481 Al Nosedal University of Toronto Spectral Analysis Winter 2016 46 / 71
Fitting Seasonal Component deaths_mean<-deaths-mean(deaths.ts); # mean corrected series; n<-length(deaths.ts); time<-c(1:n); col_1<-rep(1,n); col_c<-cos(2*pi*time/12); col_s<-sin(2*pi*time/12); X<-cbind(col_1,col_c,col_s); fit_deaths<-lm(deaths_mean~ -1 + X); # no intercept needed; # it is included in X; Al Nosedal University of Toronto Spectral Analysis Winter 2016 47 / 71
Fitting Seasonal Component summary(fit_deaths)[4] Al Nosedal University of Toronto Spectral Analysis Winter 2016 48 / 71
Fitting Seasonal Component ## $coefficients ## Estimate Std. Error t value Pr(> t ) ## Xcol_1-1.594540e-13 74.50627-2.140142e-15 1.000000e+00 ## Xcol_c -7.340397e+02 105.36778-6.966453e+00 1.529930e-09 ## Xcol_s -7.116420e+02 105.36778-6.753886e+00 3.703885e-09 Al Nosedal University of Toronto Spectral Analysis Winter 2016 49 / 71
Predictions vs Observations preds_ts<-ts(fit_deaths$fit,start=1973,freq=12); # turning predictions from linear #model into a time series object; plot.ts(deaths.ts,ylab="thousands"); lines(preds_ts+mean(deaths.ts),lty=2, col="blue"); legend("topright",c("obs", "pred"), lty=c(1,2), col=c("black","blue"),bty="n" ); Al Nosedal University of Toronto Spectral Analysis Winter 2016 50 / 71
thousands 7000 9000 11000 obs pred 1973 1975 1977 1979 Time Al Nosedal University of Toronto Spectral Analysis Winter 2016 51 / 71
Graph of deseasonalized data deaths_deseason<-fit_deaths$res; plot.ts(deaths_deseason); Al Nosedal University of Toronto Spectral Analysis Winter 2016 52 / 71
deaths_deseason 1500 0 1500 0 10 20 30 40 50 60 70 Time Al Nosedal University of Toronto Spectral Analysis Winter 2016 53 / 71
Some comments The graph of deseasonalized data suggests the presence of an additional quadratic trend function. Al Nosedal University of Toronto Spectral Analysis Winter 2016 54 / 71
Fitting Quadratic Trend n<-length(deaths_deseason); time<-c(1:n); col_1<-rep(1,n); time2<-time^2; T<-cbind(col_1,time,time2); fit_trend<-lm(deaths_deseason~ -1+T); # no intercept needed; # it is included in T; Al Nosedal University of Toronto Spectral Analysis Winter 2016 55 / 71
Fitting Quadratic Trend summary(fit_trend)[4] Al Nosedal University of Toronto Spectral Analysis Winter 2016 56 / 71
Fitting Quadratic Trend ## $coefficients ## Estimate Std. Error t value Pr(> t ) ## Tcol_1 1138.8686110 178.3411517 6.385899 1.692203e-08 ## Ttime -71.3667762 11.2743491-6.330013 2.128145e-08 ## Ttime2 0.8309979 0.1496698 5.552210 4.892800e-07 Al Nosedal University of Toronto Spectral Analysis Winter 2016 57 / 71
Plot of Residuals Res<-fit_trend$res plot.ts(res); Al Nosedal University of Toronto Spectral Analysis Winter 2016 58 / 71
Plot of Residuals Res 1000 0 1000 0 10 20 30 40 50 60 70 Time Al Nosedal University of Toronto Spectral Analysis Winter 2016 59 / 71
ACF of Residuals acf(res); Al Nosedal University of Toronto Spectral Analysis Winter 2016 60 / 71
ACF of Residuals Series Res ACF 0.4 0.2 0.8 0 5 10 15 Lag Al Nosedal University of Toronto Spectral Analysis Winter 2016 61 / 71
Some comments From our plots, it is clear that there is substantial dependence in the series of residuals. Al Nosedal University of Toronto Spectral Analysis Winter 2016 62 / 71
Modeling Residuals library(forecast); auto.arima(res); Al Nosedal University of Toronto Spectral Analysis Winter 2016 63 / 71
Modeling Residuals ## Series: Res ## ARIMA(1,0,1) with zero mean ## ## Coefficients: ## ar1 ma1 ## -0.6626 0.9264 ## s.e. 0.1164 0.0518 ## ## sigma^2 estimated as 205030: log likelihood=-542.79 ## AIC=1091.58 AICc=1091.93 BIC=1098.41 Al Nosedal University of Toronto Spectral Analysis Winter 2016 64 / 71
The classical decomposition model X t = m t + s t + Y t where m t is a slowly changing function known as a trend component, s t is a function with known period d referred to as the seasonal component, and Y t is a random noise component that is stationary. Al Nosedal University of Toronto Spectral Analysis Winter 2016 65 / 71
The classical decomposition model For our example, we have that X t 8787.7361111 734.0397113cos( 2πt 2πt 12 ) 711.6419859sin( 12 ) +1138.868611 + 71.3667762t + 0.8309979t 2 0.6625909Y t 1 0.6625909W t 1 + W t Al Nosedal University of Toronto Spectral Analysis Winter 2016 66 / 71
Another way auto.arima(deaths.ts); Al Nosedal University of Toronto Spectral Analysis Winter 2016 67 / 71
Another way ## Series: deaths.ts ## ARIMA(0,1,1)(0,1,1)[12] ## ## Coefficients: ## ma1 sma1 ## -0.4264-0.5584 ## s.e. 0.1226 0.1787 ## ## sigma^2 estimated as 99480: log likelihood=-425.53 ## AIC=857.06 AICc=857.5 BIC=863.3 Al Nosedal University of Toronto Spectral Analysis Winter 2016 68 / 71
Forecasts sarima<-auto.arima(deaths.ts); fcast<-forecast(sarima); plot(fcast); Al Nosedal University of Toronto Spectral Analysis Winter 2016 69 / 71
Forecasts Forecasts from ARIMA(0,1,1)(0,1,1)[12] 6000 10000 1974 1976 1978 1980 Al Nosedal University of Toronto Spectral Analysis Winter 2016 70 / 71
Some final comments The corresponding fitted model for {X t } is thus the SARIMA (0, 1, 1) (0, 1, 1) 12 process where {W t } WN(0, σ 2 W = 99480). (1 0.4264B)(1 0.5584B 12 )W t, Al Nosedal University of Toronto Spectral Analysis Winter 2016 71 / 71