Decision 411: Class 3

Decision 411: Class 3 Discussion of HW#1 Introduction to seasonal models Seasonal decomposition Seasonal adjustment on a spreadsheet Forecasting with seasonal adjustment Forecasting inflation Log transformation = poor man s deflator Confidence intervals for composite forecasts Seasonality A repeating, periodic pattern in the data that is keyed to the calendar or the clock Not the same as cyclicality, i.e., business cycle effects, which do not have a predictable periodicity Most regular seasonal patterns have an annual period (12 months or 4 quarters), although other possibilities exist: Day-of of-week effects (period=5 or 7 days) End-of of-quarter effects (period = 3 months) 1

Examples Weather-related related demand Heating oil, electricity, snow shovels... Holiday purchasing Christmas, Easter, Super Sunday... Seasonal tourism Winter skiing, summer vacations... Academic year Back-to to-school clothing, books, shoes... Seasonal patterns are complex, because the calendar is not rational Retail activity is geared to the business week, but months and years do not have whole numbers of weeks A given month does not always have the name number of trading days or weekends Some major holidays (e.g., Easter) are moveable feasts that do not occur on the same dates each year 2

Quarterly vs. monthly vs. weekly Quarterly data are easiest to handle: 4 quarters in a year, 3 months in a quarter, trading day adjustments are minimal Monthly data are more complicated: 12 months in a year, but not 4 weeks in a month; trading day adjustments may be important Weekly data require special handling because a year is not exactly 52 weeks How to model seasonal patterns Seasonal adjustment of the data (today) Seasonally adjust the original data Fit a forecasting model to the adjusted data Re-seasonalize seasonalize the forecasts Seasonal dummy variables (regression) Seasonal lags and differences (ARIMA) 3

Seasonal adjustment An additive or multiplicative adjustment of the data to correct for the anticipated effects of seasonality Two uses for seasonal adjustment To provide a different view of the data that reveals underlying trends apart from normal seasonal effects As a component of a forecasting model in which a non-seasonal model is fitted to seasonally adjusted data Caveats about seasonal modeling Be sure there is a seasonal pattern before trying to fit a seasonal model Seasonal adjustment adds many parameters to the model and carries a risk of overfitting overfitting if a seasonal pattern is weak or absent altogether The risk of overfitting is reduced if seasonal indices are estimated on aggregated data (rather than 1000 separate products) In some cases, it may also be advisable to shrink seasonal indices toward 100% to introduce a note of conservatism 4

Multiplicative seasonality Most natural seasonal patterns are multiplicative: Seasonal variations are roughly constant in percentage terms Seasonal swings therefore get larger or smaller in absolute magnitude as the average level of the series rises or falls due to long-term trends and/or business cycle effects Additive seasonality A log transformation converts a multiplicative pattern to an additive one: An additive seasonal pattern has constant- amplitude seasonal swings even in the presence of trends and cycles If your model includes a log transformation,, use additive rather than multiplicative seasonal adjustment 5

Seasonal modeling in Statgraphics When entering the Time Series procedures, enter a value for the seasonal period in the Seasonality box to activate seasonal options The sampling interval & starting date merely affect the labeling of the plots, not the analysis itself Entering a number here activates the seasonal options (use 12 for monthly, 4 for quarterly, etc.) Descriptive Methods procedure The time series plot will show if there is an obvious seasonal pattern The autocorrelation plot provides a more sensitive test Check to see if there is significant autocorrelation at the seasonal period (e.g., lag 4 for quarterly data, lag 12 for monthly data) If the series has a strong trend, it helps to de-trend it or take a non-seasonal difference before looking for seasonal autocorrelation 6

Here the seasonal pattern is apparent in both the time series plot and the autocorrelation plot but there is also strong positive autocorrelation at all lower-order lags due to an upward linear trend De-trending The right-mouse-button Analysis options let you de-trend the series in various ways. Here we ll use the linear trend option because the trend appears roughly linear on the time series plot 7

After linearly de-trending, the seasonal autocorrelation (at lags 12, 24, etc.) stands out more strongly. There is also a small but systematic pattern at intermediate lags, but these are not important individually they are side-effects of the shape of the overall seasonal pattern. Seasonal decomposition Seasonal decomposition means decomposing a series at each time t into: a trend-cycle component T t a seasonal component S t an irregular (random, unexplained) component I t where appropriate, a trading day adjustment D t 8

Seasonal decomposition This is historically the oldest method of time series analysis intuitive intuitive but ad hoc The Census Bureau has an elaborate seasonal decomposition program called X-12 ARIMA : http://www.census.gov/srd/www/x12a/x12down_pc.html Statgraphics uses a simpler approach that can also be implemented on a spreadsheet Seasonal decomposition formulas The components of a series are usually assumed to interact multiplicatively: With trading day adjustment: Y t = T t S t D t I t Without trading day adjustment: Y t = T t S t I t The seasonally adjusted series is the original series divided by the seasonal component(s): Y t S t D t or Y t S t 9

Seasonal Indices The seasonal index S t represents the expected percentage of normal in the season (e.g., month or quarter) that corresponds to time t For example, if the January index is 89, this means that January s value is expected to be 89% of normal, where normal is defined by the monthly average for the whole surrounding year. In this case, January s seasonally adjusted value would be the actual value divided by 0.89, thus scaling up the actual value a bit to correct for expected below-normal levels in January Seasonal indices, continued When the seasonal indices are assumed to be stable over time, they can be estimated by the ratio to moving average method, as in Statgraphics. Time-varying seasonal indices can also be estimated with the Census Bureau s X-12 X program or Winter s seasonal exponential smoothing model 10

Seasonal decomposition by the ratio-to to- moving average method Step 1: determine the trend-cycle component by computing a one-year centered moving average (losing 1/2 year of data at either end*) Step 2: calculate the ratio of the original series to the moving average at each point to determine the % deviation from normal Step 3: average the ratios by season (e.g., average all the January ratios, then all the February ratios, etc.) *For this reason X-12 X uses forward/backward forecasting Seasonal decomposition by the ratio-to to- moving average method, continued Step 4: Renormalize the ratios (if necessary) so they add up exactly to the number of periods in year Step 5: The seasonally adjusted series is the original series divided by the seasonal indices Step 6: The irregular component is the seasonally adjusted series divided by the trend-cycle component 11

Quarterly data: sales at Outboard Marine 530 Time Series Plot for SALES SALES 430 330 230 130 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 Seasonal adjustment on a spreadsheet 12

Plot of moving average and seasonally adjusted values 600.0 500.0 400.0 300.0 200.0 100.0 0.0 Dec-83 Jun-84 Dec-84 Jun-85 Dec-85 Jun-86 Dec-86 Jun-87 Dec-87 Jun-88 Dec-88 Jun-89 Dec-89 Jun-90 Dec-90 Jun-91 Dec-91 Jun-92 Dec-92 Jun-93 Original data Moving average Seasonally adjusted Note that the centered moving average is also a seasonally adjusted view of the data, but it is also heavily smoothed both forward and backward in time. Seasonal adjustment of Outboard Marine data in Statgraphics In the Describe/Time Series/ Seasonal Decomposition procedure you can see the various components of the seasonal adjustment process. 13

Trend-cycle component = 1-year centered moving average Trend-Cycle Component Plot for SALES SALES 530 430 330 230 data trend-cycle 130 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 Seasonal indices = (normalized) averages of ratios-to to-moving-average 118 Seasonal Index Plot for SALES seasonal index 108 98 88 78 68 0 1 2 3 4 5 season 14

Seasonally adjusted values = data divided by seasonal indices Seasonally Adjusted Data Plot for SALES seasonally adjusted 440 400 360 320 280 240 200 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 Irregulars = seasonally adjusted values divided by trend-cycle 117 Irregular Component Plot for SALES irregular 112 Here too, the 107 seasonal pattern 102 seems to have changed, as 97 indicated by larger 92 irregulars at ends of series 87 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 15

Seasonal Decomposition Procedure: tables and charts Trend-cycle component Seasonal Indices Irregular component Seasonally adjusted data Seasonal subseries plot Annual subseries plot Seasonal decomposition procedure 16

Trend-cycle component is estimated by a 12-month centered moving average Trend-Cycle Component Plot for RetailxAuto/CPI RetailxAuto/CPI 1630 1430 1230 1030 Dec. 92: 129% of normal 830 1/92 1/94 1/96 1/98 1/00 1/02 data trend-cycle Dec. 98: 126% of normal The trend-cycle component is a smoothed estimate of the normal level of the series at each point Seasonal indices are estimated by averaging the ratio of original series to the moving average by season (month) Seasonal Index Plot for RetailxAuto/CPI seasonal index 136 126 116 106 96 Average fraction of normal in December is 127.6% 86 0 3 6 9 12 15 season 17

Seasonally adjusted series is the original series divided by the seasonal indices Seasonally Adjusted Data Plot for RetailxAuto/CPI seasonally adjusted 1330 1230 1130 1030 930 1/92 1/94 1/96 1/98 1/00 1/02 Irregular component is the seasonally adjusted series divided by the trend-cycle component Irregular Component Plot for RetailxAuto/CPI irregular 105 103 101 99 February 2000 97 1/92 1/94 1/96 1/98 1/00 1/02 18

This plot shows whether the same trends have been observed in each season i i.e., have some seasons grown more than others? RetailxAuto/CPI 1630 1430 1230 1030 Seasonal Subseries Plot for RetailxAuto/CPI 830 0 2 4 6 8 10 12 14 Season This plot can be used to help determine whether seasonal indices are actually constant over time, and it also highlights the comparison of different seasons. This plot shows whether the seasonal pattern has looked roughly the same in each year (here, it has) RetailxAuto/CPI 1630 1430 1230 1030 830 Annual Subseries Plot for RetailxAuto/CPI 0 2 4 6 8 10 12 Season Cycle 1 2 3 4 5 6 7 8 9 10 11 This plot also sheds light on the question of whether the seasonal pattern is contant, and it also highlights the comparison of different years. 19

Seasonally adjusted series published by government (blue), based on X-12 X program, is much smoother than the one computed in Statgraphics (red), due to trading day adjustments 1330 Variables RetailxAutoSA/CPI SADJUSTED 1230 1130 1030 930 1992 1994 1996 1998 2000 2002 TIME What does X-12 X do that Statgraphics doesn t? It adjusts for trading days and uses forward and backward forecasting to avoid data loss at ends of series It begins by automatically fitting an ARIMA* model with regression variables to adjust for trading days, trends, etc., and uses it to forecast both forward and backward. Short-term term tapered moving averages are then used to estimate time-varying seasonal indices on the extended data. *To be discussed in week 5. 20

Example of a changing seasonal pattern: bookstore sales BookstoreSales/BookCPI Time Series Plot for BookstoreSales/BookCPI 8 6 4 2 0 1/72 1/77 1/82 1/87 1/92 1/97 1/02 30-year history of bookstore sales Trend-cycle component shows irregular overall growth 21

but growth has been larger in some months than others Trend-Cycle Component Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 1/72 1/77 1/82 1/87 1/92 1/97 1/02 data trend-cycle January values highlighted in red Trend-Cycle Component Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 1/72 1/77 1/82 1/87 1/92 1/97 1/02 data trend-cycle August values highlighted in red 22

Irregular Component Plot for BookstoreSales/BookCPI 149 irregular 129 109 89 69 1/72 1/77 1/82 1/87 1/92 1/97 1/02 Dumbell pattern in the irregular plot shows that the seasonal pattern is best fitted in the middle of the series, suggesting that it has changed over time Seasonally Adjusted Data Plot for BookstoreSales/BookCPI seasonally adjusted 8 6 4 2 0 1/72 1/81 1/90 1/99 1/08 Plot of seasonally adjusted data also suggests that fixed-index seasonal adjustment hasn t worked well at either end of the series 23

Seasonal Subseries Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 0 3 6 9 12 Seasonal 15 subseries plot Season shows dramatically that the seasonal variation in January, August, and December has become much larger over the years Annual Subseries Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 Cycle 1 2 3 4 5 6 7 8 9 10 0 3 6 9 12 15Annual 11 subseries 12 Season plot also shows 13 increasing variation 14 in January, August, and December 24

Time Series Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8.7 7.7 6.7 5.7 4.7 3.7 Data since 1992 still shows a changing pattern (January highlighted in red) 2.7 1/92 1/94 1/96 1/98 1/00 1/02 Data since 1992 still shows a variable seasonal pattern Trend-Cycle Component Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8.7 7.7 6.7 5.7 4.7 3.7 2.7 1/92 1/94 1/96 1/98 1/00 1/02 data trend-cycle January values circled 25

Seasonal Subseries Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8.7 7.7 6.7 5.7 4.7 3.7 2.7 0 3 6 9 12 15 Season Seasonal subseries plot confirms that the trend in January has been different from other months What to do when seasonal patterns change? Use a shorter data history e.g., last 4 or 5 seasons but beware of overfitting! Estimate time-varying seasonal indices (e.g., with X-12 X software or Winters model) Use a seasonal ARIMA model (which we ll meet later) 26

Forecasting with seasonal adjustment Seasonally adjust the data Forecast the seasonally adjusted series (e.g., with a random walk, linear trend, or exponential smoothing model) Re-seasonalize seasonalize the forecasts and confidence limits by multiplying by the appropriate seasonal indices Statgraphics does this automatically when seasonal adjustment is used as a model option The Forecasting procedure will perform seasonal adjustment, but it does not print out the seasonal indices! If you want to see them, you must use the Describe/Time Series/Seasonal Decomposition procedure separately. 27

RetailxAuto/CPI 2000 1800 1600 1400 1200 1000 Time Sequence Plot for RetailxAuto/CPI Random walk with drift 800 1/92 1/94 1/96 1/98 1/00 1/02 1/04 actual forecast 50.0% limits Reseasonalized forecasts from random walk model with drift Saving results to spreadsheet Save to datasheet A if you want to keep everything in one file Check the boxes for the model outputs you wish to save Default names for saved variables (new columns to be created on spreadsheet) The Save results icon on Analysis Window Toolbar (4th from left) can be used to save forecasts and other results to the data spreadsheet, where they can be used in calculations with other variables. Here the forecasts and limits for RetailxAuto/CPI are being saved. 28

Forecasting inflation To re-inflate forecasts of a deflated series, it is necessary to multiply the forecasts and confidence limits by a forecast of the price index The price index forecast can be obtained from a random-walk walk-with-drift model (based on recent history) or expert consensus Reinflation calculations can be performed with Statgraphics formulas or in Excel CPI 190 180 170 160 150 140 Simple inflation forecast Time Sequence Plot for CPI Random walk with drift actual forecast 50.0% limits 130 1/92 1/95 1/98 1/01 1/04 1/07 Note that there is some error in the forecast of the inflation rate, although it is small in comparison to the error in forecasting deflated sales. 29

Save (more) results to spreadsheet Now let s save the CPI forecasts to the spreadsheet Personalize the name of the saved variable (here by appending CPI at the front) so as not to conflict with previously used names Save to datasheet A if you want to keep all your results in one file Re-inflated forecasts & CI s (X 10000) 35 31 27 23 19 15 Variables RetailxAuto FORECASTS*CPIFORECASTS LLIMITS*CPIFORECASTS ULIMITS*CPIFORECASTS 11 1990 1993 1996 1999 2002 2005 DATEINDEX Here the forecasts and confidence limits for deflated sales have been multiplied by the CPI forecasts (ignoring error in CPI forecast*) *We ll come back to this issue later Alas, there is no multiple-time-series plot procedure in Statgraphics. This plotwas drawn by creating the DATEINDEX variable using the formula YEAR+(MONTH-1)/12 and then using the Plot/Scatterplot/Multiple X-Y Plot procedure to plot the data, forecasts, and CI s versus DATEINDEX. 30

The poor man s deflator Alternatively, you can use the log transformation, the poor man s deflator Logging does not remove inflation, but linearizes its effects Thus, when the data are logged, the effect of inflation merely gets lumped together with other sources of trends that are fitted by drift or trend factors in the forecasting model. Statgraphics automatically unlogs unlogs the final forecasts and CI s when logging is chosen as a model option Original retail data (not deflated) (X 10000) 29 RetailxAuto 26 23 20 17 14 Time Series Plot for RetailxAuto 11 2/92 2/94 2/96 2/98 2/00 2/02 Note that the seasonal pattern is multiplicative: seasonal swings are larger at the end of the series 31

Logged retail data Time Series Plot for adjusted RetailxAuto adjusted RetailxAuto 12.6 12.4 12.2 12 11.8 11.6 2/92 2/94 2/96 2/98 2/00 2/02 The seasonal pattern is now additive (seasonal swings have roughly constant amplitude) When a log transformation is used, the seasonal adjustment should be additive 32

(X 10000) 35 RetailxAuto 31 27 23 19 15 Unlogged forecasts Time Sequence Plot for RetailxAuto Random walk with drift 11 1/92 1/94 1/96 1/98 1/00 1/02 1/04 actual forecast 50.0% limits With a log transformation and additive adjustment, the confidence limits are now asymmetric (errors are now assumed to be lognormally distributed) Comparison of methods (X 10000) 31 Variables RetailxAuto FORECASTS*CPIFORECASTS UNLOGFORECASTS 26 21 16 1999 2001 2003 2005 DATEINDEX Here the re-inflated point forecasts from the first model are plotted alongside the unlogged point forecasts from the second model. The results are virtually identical because both models assume that the rate of inflation has been relatively constant over this period. However, the confidence intervals (not shown here) are slightly different because of the normal-vs-lognormal issue. 33

When to log, when to deflate? Deflation should be used when you are interested in knowing the forecast in real terms and/or if the inflation rate is expected to change Logging is sufficient if you just want a forecast in nominal terms and inflation is expected to remain constant inflation just gets lumped with other sources of trends in the model. Logging also ensures that forecasts and confidence limits have positive values,, even in the presence of downward trends and/or high volatility. If inflation has been minimal and/or there is little overall trend, neither may be necessary Outboard Marine revisited SALES 530 430 330 230 Time Sequence Plot for SALES Random walk with drift actual forecast 50.0% limits 130 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 Here no deflation or logging was used: sales are flat in current dollars, declining in real terms 34

Outboard Marine revisited SALES 530 430 330 230 Time Sequence Plot for SALES Random walk with drift actual forecast 50.0% limits 130 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 Same model, except with natural log transformation and additive adjustment (point forecasts are about the same, confidence limits now asymmetric) Confidence limits for composite forecasts Suppose you forecast X and Y separately, but you are really interested in X+Y or X Y. X It is OK to add or multiply the corresponding point forecasts,, but can you add or multiply the corresponding confidence limits?? NO!!! If the errors in forecasting X and Y are independent (i.e., uncorrelated), there are ways to approximate the confidence limits. 35

Confidence limits for composite forecasts: the basic principle In the case of a sum of forecasts, the variances of the forecast errors are additive*. In the case of a product of forecasts, the variances of the percentage forecast errors are (approximately) additive.* ** These relations lead to square root of sum of squares formulas for the standard errors and confidence limits. *Assuming statistical independence of X and Y errors **Strictly speaking, variances are additive in logged units Example of sum of forecasts Suppose the forecast and CI for X is 20 ± 3 while the forecast and CI for Y is 30 ± 4 (for some specified level of confidence) Then the corresponding forecast and CI for X+Y is: (20+ 30) ± 3 2 + 4 2 = 50± 5 36

Example of product of forecasts Suppose the forecast and CI for X is 20 ± 3% while the forecast and CI for Y is 30 ± 4% Then the forecast and CI for X Y X Y is: (20 30) ± 3% 2 + 4% 2 = 600± 5% Note that it is necessary to translate CI s into percentage terms before applying the product formula Retail sales revisited: confidence limits for the re-inflated forecasts Forecast & 50% CI for RetailxAuto/CPI in next period (March 02) is 125138 ± 1.80% Forecast & CI for CPI is 1.7775 ± 0.175% Forecast for RetailxAuto (i.e., their product) ) is: 2 2 (125138 1.7775) ± 1.80% + 0.175% = 222433± 1.81% Notice that because the % error in the CPI forecast is smaller by a factor of 1/10, which becomes a factor of 1/100 when squared, it can essentially be ignored in this case. 37

Are the errors really independent? The actual correlation between errors for RetailxAuto/CPI and CPI is only -0.01: This report was obtained by saving the residuals of both models to the data spreadsheet, then running the Describe/Numeric Data/Multiple- Variable procedure to get the correlation matrix This correlation is not significantly different from zero, so the approximation for the CI s is OK Positive correlation would imply wider CI s, negative correlation would imply narrower CI s For next time Read handout on exponential smoothing Watch video clips #10-14 14 HW#2 is due a week from today 38