Decision 411: Class 3 Discussion of HW#1 Introduction to seasonal models Seasonal decomposition Seasonal adjustment on a spreadsheet Forecasting with seasonal adjustment Forecasting inflation Poor man s deflator Confidence intervals for composite forecasts
Seasonality Refers to a repeating, periodic pattern in the data that is keyed to the calendar or the clock Not the same as cyclicality, which does not have a fixed period Most seasonal patterns have an annual period (12 months or 4 quarters), although other possibilities exist Day-of of-week effects (period=5 or 7 days) End-of of-quarter effects (period = 3 months)
Examples Weather-related related demand Heating oil, electricity, snow shovels... Holiday purchasing Christmas, Easter, Super Sunday... Seasonal tourism Winter skiing, summer vacations... Academic year Back-to to-school clothing, books, shoes...
Seasonal patterns are complex, because the calendar is not rational Retail activity is geared to the business week, but months and years do not have whole numbers of weeks A given month does not always have the name number of trading days or weekends Some major holidays (e.g., Easter) are moveable feasts that do not occur on the same dates each year
Quarterly vs. monthly vs. weekly Quarterly data are easiest to handle: 4 quarters in a year, 3 months in a quarter, trading day adjustments are minimal Monthly data are more complicated: 12 months in a year, but not 4 weeks in a month; trading day adjustments may be important Weekly data require special handling because a year is not exactly 52 weeks
How to model seasonal patterns Seasonal adjustment of the data (today) Seasonally adjust the original data Fit a forecasting model to the adjusted data Re-seasonalize seasonalize the forecasts Seasonal dummy variables (regression) Seasonal lags and differences (ARIMA)
Seasonal adjustment An additive or multiplicative adjustment of the data to correct for the anticipated effects of seasonality Two uses for seasonal adjustment To provide a different view of the data that reveals underlying trends apart from normal seasonal effects As a component of a forecasting model in which a non-seasonal model is fitted to seasonally adjusted data
Caveats about seasonal modeling Be sure there is a seasonal pattern before trying to fit a seasonal model Seasonal adjustment adds many parameters to the model and carries a risk of overfitting The risk of overfitting is reduced if seasonal indices are estimated on aggregated data (rather than 10,000 separate products) In some cases, it may also be advisable to shrink seasonal indices toward 100% to introduce a note of conservatism
Types of seasonal patterns Most natural seasonal patterns are multiplicative Seasonal variations are constant in percentage terms Seasonal swings therefore get larger as the level of the series rises or falls due to trends and cycles
Types of seasonal patterns A log transformation converts a multiplicative pattern to an additive one An additive seasonal pattern has constant- amplitude seasonal swings even in the presence of a trend If your model includes a log transformation,, use additive rather than multiplicative seasonal adjustment
Seasonal modeling in Statgraphics When entering the Time Series procedures, enter a value for the seasonal period in the Seasonality box to activate seasonal options The sampling interval & starting date merely affect the labeling of the plots, not the analysis itself Entering a number here activates the seasonal options (use 12 for monthly, 4 for quarterly, etc.)
Descriptive Methods procedure The time series plot will show if there is an obvious seasonal pattern The autocorrelation plot provides a more sensitive test Check to see if there is significant autocorrelation at the seasonal period (e.g., lag 12 for monthly data) If the series has a strong trend, it helps to de- trend it or take a non-seasonal difference before looking for seasonal autocorrelation
Here the seasonal pattern is apparent in both the time series plot and the autocorrelation plot, but there is also strong autocorrelation at all lags due to the upward trend
De-trending The right-mouse-button Analysis options let you de-trend the series
After de-trending, the seasonal autocorrelation (at lags 12, 24, etc.) stands out more strongly
Seasonal decomposition Seasonal decomposition means decomposing a series into a trend-cycle component (T( t ) a seasonal component (S( t ) an irregular component (I( t ) (if appropriate) a trading day adjustment (D( t )
Seasonal decomposition This is historically the oldest method of time series analysis-- --intuitive but ad hoc The Census Bureau has an elaborate seasonal decomposition program called X-12 ARIMA : http://www.census.gov/srd/www/x12a/x12down_pc.html Statgraphics uses a simpler approach that can also be implemented on a spreadsheet
Seasonal decomposition, continued The components of a series are usually assumed to interact multiplicatively: With trading day adjustment: Y t = T t S t D t I t Without trading day adjustment: Y t = T t S t I t The seasonally adjusted series is the original series divided by the seasonal component: Y t /(S t D t ) or Y t /S t
Seasonal Indices The seasonal component consists of seasonal indices representing the expected percentage of normal in a given season (e.g., in a month or quarter) For example, if January s index is 89, this means that January s value is expected to be 89% of normal, where normal is defined by the monthly average for the whole year. In this case, January s seasonally adjusted value would be the actual value divided by 0.89
Seasonal indices, continued When the seasonal indices are assumed to be stable over time, they can be estimated by the ratio to moving average method, as in Statgraphics. Time-varying seasonal indices can also be estimated (as in the Census Bureau s X-12 X program and Winter s seasonal exponential smoothing model)
Seasonal decomposition by the ratio-to to- moving average method Step 1: determine the trend-cycle component by computing a one-year centered moving average (losing 1/2 year of data at either end*) Step 2: calculate the ratio of the original series to the moving average at each point to determine the % deviation from normal Step 3: average the ratios by season (e.g., average all the January ratios, then all the February ratios, etc.) *For this reason X-12 X uses forward/backward forecasting
Seasonal decomposition by the ratio-to to- moving average method, continued Step 4: Renormalize the ratios (if necessary) so they add up exactly to the number of periods in year Step 5: The seasonally adjusted series is the original series divided by the seasonal indices Step 6: The irregular component is the seasonally adjusted series divided by the trend-cycle component
Quarterly data: sales at Outboard Marine 530 Time Series Plot for SALES SALES 430 330 230 130 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98
Seasonal adjustment on a spreadsheet
Seasonal adjustment on a spreadsheet Seasonal adjustment on a spreadsheet 600.0 500.0 400.0 Original data 300.0 Moving average Seasonally adjusted 200.0 100.0 0.0 Dec-83 Jun-84 Dec-84 Jun-85 Dec-85 Jun-86 Dec-86 Jun-87 Dec-87 Jun-88 Dec-88 Jun-89 Dec-89 Jun-90 Dec-90 Jun-91 Dec-91 Jun-92 Dec-92 Jun-93
Seasonal adjustment in Statgraphics In the Describe/Time Series/ Seasonal Decomposition procedure you can see the various components of the seasonal adjustment process.
Trend-cycle component = 1-year centered moving average Trend-Cycle Component Plot for SALES SALES 530 430 330 230 data trend-cycle 130 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98
Seasonal indices = (normalized) averages of ratios-to to-moving-average 118 Seasonal Index Plot for SALES seasonal index 108 98 88 78 68 0 1 2 3 4 5 season
Seasonally adjusted values = data divided by seasonal indices Seasonally Adjusted Data Plot for SALES seasonally adjusted 440 400 360 320 280 240 200 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98
Irregulars = seasonally adjusted values divided by trend-cycle 117 Irregular Component Plot for SALES irregular 112 107 102 97 92 87 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 Here too, the seasonal pattern seems to have changed, as indicated by larger irregulars at ends of series
Seasonal Decomposition Procedure Trend-cycle component Seasonal Indices Irregular component Seasonally adjusted data Seasonal subseries Annual subseries
Seasonal decomposition procedure
Trend-cycle component is estimated by a 12-month centered moving average Trend-Cycle Component Plot for RetailxAuto/CPI RetailxAuto/CPI 1630 1430 1230 1030 Dec. 92: 129% of normal 830 1/92 1/94 1/96 1/98 1/00 1/02 data trend-cycle Dec. 98: 126% of normal The trend-cycle component is a smoothed estimate of the normal level of the series at each point
Seasonal indices are estimated by averaging the ratio of original series to the moving average by season (month) Seasonal Index Plot for RetailxAuto/CPI seasonal index 136 126 116 106 96 Average fraction of normal in December is 127.6% 86 0 3 6 9 12 15 season
Seasonally adjusted series is the original series divided by the seasonal indices Seasonally Adjusted Data Plot for RetailxAuto/CPI seasonally adjusted 1330 1230 1130 1030 930 1/92 1/94 1/96 1/98 1/00 1/02
Irregular component is the seasonally adjusted series divided by the trend-cycle component Irregular Component Plot for RetailxAuto/CPI irregular 105 103 101 99 February 2000 97 1/92 1/94 1/96 1/98 1/00 1/02
This plot shows whether the same trends have been observed in each season-- --i.e., have some seasons grown more than others? 1630 Seasonal Subseries Plot for RetailxAuto/CPI RetailxAuto/CPI 1430 1230 1030 830 0 2 4 6 8 10 12 14 Season
This plot shows whether the seasonal pattern has looked roughly the same in each year (here, it has) Annual Subseries Plot for RetailxAuto/CPI RetailxAuto/CPI 1630 1430 1230 1030 830 0 2 4 6 8 10 12 Season Cycle 1 2 3 4 5 6 7 8 9 10 11
Seasonally adjusted series published by government (blue), based on X-12 X program, is much smoother than the one computed in Statgraphics (red), due to trading day adjustments 1330 Variables RetailxAutoSA/CPI SADJUSTED 1230 1130 1030 930 1992 1994 1996 1998 2000 2002 TIME
What does X-12 X do that Statgraphics doesn t? It adjusts for trading days and uses forward and backward forecasting to avoid data loss at ends of series It begins by automatically fitting an ARIMA model with regression variables to adjust for trading days, trends, etc., and uses it to forecast both forward and backward. Short-term term tapered moving averages are then used to estimate time-varying seasonal indices on the extended data.
Seasonal patterns can change over time Time Series Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 1/72 1/77 1/82 1/87 1/92 1/97 1/02 30-year history of bookstore sales
Seasonal patterns can change over time
Seasonal patterns can change over time Trend-Cycle Component Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 1/72 1/77 1/82 1/87 1/92 1/97 1/02 data trend-cycle January values highlighted in red
Trend-Cycle Component Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 1/72 1/77 1/82 1/87 1/92 1/97 1/02 data trend-cycle August values highlighted in red
Irregular Component Plot for BookstoreSales/BookCPI 149 irregular 129 109 89 69 1/72 1/77 1/82 1/87 1/92 1/97 1/02 Dumbell pattern in the irregular plot shows that the seasonal pattern is best fitted in the middle of the series, suggesting that it has changed over time
Seasonally Adjusted Data Plot for BookstoreSales/BookCPI seasonally adjusted 8 6 4 2 0 1/72 1/81 1/90 1/99 1/08 Plot of seasonally adjusted data also suggests that fixedindex seasonal adjustment hasn t worked well at either end of the series
Seasonal Subseries Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 Seasonal subseries plot shows dramatically that the seasonal variation in January, August, and December has become much larger 0 3 6 9 12 15 over the years Season
Annual Subseries Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8 6 4 2 0 0 3 6 9 12 15 Season Cycle 1 2 3 4 Annual 5 subseries 6plot also shows 7 increasing 8 variation in January, 9 August, 10and December 11 12 13 14
BookstoreSales/BookCPI Time Series Plot for BookstoreSales/BookCPI 8.7 7.7 6.7 5.7 4.7 3.7 2.7 1/92 1/94 1/96 1/98 1/00 1/02 Data since 1992 Data still shows since a 1992 changing still shows pattern a variable (January seasonal highlighted pattern in red)
Trend-Cycle Component Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8.7 7.7 6.7 5.7 4.7 3.7 2.7 1/92 1/94 1/96 1/98 1/00 1/02 data trend-cycle January values circled
Seasonal Subseries Plot for BookstoreSales/BookCPI BookstoreSales/BookCPI 8.7 7.7 6.7 5.7 4.7 3.7 2.7 0 3 6 9 12 15 Season Seasonal subseries plot confirms that the trend in January has been different from other months
What to do when seasonal patterns change? Use a shorter data history (e.g., last 4 or 5 years but beware of overfitting!) Estimate time-varying seasonal indices (e.g., with X-12 X software or Winters model) Use a seasonal ARIMA model (which we ll meet later)
Forecasting with seasonal adjustment Seasonally adjust the data Forecast the seasonally adjusted series (e.g., with a random walk, linear trend, or exponential smoothing model) Re-seasonalize seasonalize the forecasts and confidence limits by multiplying by the appropriate seasonal indices Statgraphics does this automatically when seasonal adjustment is used as a model option
The Forecasting procedure will perform seasonal adjustment, but it does not print out the seasonal indices! If you want to see them, you must use the Seasonal Decomposition procedure separately.
RetailxAuto/CPI Time Sequence Plot for RetailxAuto/CPI 2000 1800 1600 1400 1200 1000 Random walk with drift 800 1/92 1/94 1/96 1/98 1/00 1/02 1/04 actual forecast 50.0% limits Reseasonalized forecasts from random walk model with drift
Saving results to spreadsheet Save to datasheet A if you want to keep everything in one file Check the boxes for the model outputs you wish to save Default names for saved variables (new columns to be created on spreadsheet) The Save results icon on Analysis Window Toolbar (4th from left) can be used to save forecasts and other results to the data spreadsheet, where they can be used in calculations with other variables. Here the forecasts and limits for RetailxAuto/CPI are being saved.
Forecasting inflation To re-inflate forecasts of a deflated series (when appropriate), it is necessary to multiply the forecasts and confidence limits by a forecast of the price index The price index forecast can be obtained from a random-walk walk-with-drift model (based on recent history) or expert consensus Reinflation calculations can be performed with Statgraphics formulas or in Excel
Simple inflation forecast 190 180 170 Time Sequence Plot for CPI Random walk with drift actual forecast 50.0% limits CPI 160 Note that there is 150 some error in the forecast of the 140 inflation rate, although it is small 130 in comparison to 1/92 1/95 1/98 1/01 1/04 1/07 the error in forecasting deflated sales.
Save (more) results to spreadsheet Now let s save the CPI forecasts to the spreadsheet Personalize the name of the saved variable (here by appending CPI at the front) so as not to conflict with previously used names Save to datasheet A if you want to keep all your results in one file
Re-inflated forecasts & CI s (X 10000) 35 31 27 23 19 15 Variables RetailxAuto FORECASTS*CPIFORECASTS LLIMITS*CPIFORECASTS ULIMITS*CPIFORECASTS Here the forecasts and confidence limits for deflated sales have been multiplied by the CPI forecasts (ignoring error in CPI forecast*) 11 1990 1993 1996 1999 2002 2005 DATEINDEX *We ll come back to this issue later
Poor man s deflator Alternatively, you can use the log transformation, the poor man s deflator Logging does not remove inflation, but linearizes its effects When the data are logged, inflation is lumped together with other linear trend factors Statgraphics automatically unlogs unlogs forecasts when logging is chosen as a model option
Original retail data (not deflated) (X 10000) 29 Time Series Plot for RetailxAuto RetailxAuto 26 23 20 17 14 11 2/92 2/94 2/96 2/98 2/00 2/02 Note that the seasonal pattern is multiplicative: seasonal swings are larger at the end of the series
Logged retail data Time Series Plot for adjusted RetailxAuto adjusted RetailxAuto 12.6 12.4 12.2 12 The seasonal 11.8 pattern is now additive 11.6 (seasonal 2/92 2/94 2/96 2/98 2/00 2/02 swings have roughly constant amplitude)
When a log transformation is used, the seasonal adjustment should be additive
Unlogged forecasts (X 10000) 35 RetailxAuto 31 27 23 19 15 Time Sequence Plot for RetailxAuto Random walk with drift 11 1/92 1/94 1/96 1/98 1/00 1/02 1/04 actual forecast 50.0% limits With log transformation and additive adjustment, the confidence limits are now asymmetric (errors are assumed to be lognormal)
Comparison of methods (X 10000) 31 Variables RetailxAuto FORECASTS*CPIFORECASTS UNLOGFORECASTS 26 21 16 1999 2001 2003 2005 DATEINDEX Point forecasts are nearly identical! (Both models make similar assumptions about constant inflation.)
When to log, when to deflate? Deflation should be used when you are interested in knowing the forecast in real terms and/or if the inflation rate is expected to change Logging is sufficient if you just want a forecast in nominal terms and inflation is expected to remain constant inflation just gets lumped with other sources of trend in the model. Logging also ensures that forecasts and confidence limits have positive values,, even in the presence of downward trends and/or high volatility. If inflation has been minimal and/or there is little overall trend, neither may be necessary
Outboard Marine revisited SALES 530 430 330 230 Time Sequence Plot for SALES Random walk with drift 130 Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 actual forecast 50.0% limits Here no deflation or logging was used: sales are flat in current dollars, declining in real terms
Outboard Marine revisited SALES 530 430 Time Sequence Plot for SALES Random walk with drift actual forecast 50.0% limits 330 Same model, except 230 with natural log transformation and additive adjustment 130 (point forecasts are Q4/83 Q4/86 Q4/89 Q4/92 Q4/95 Q4/98 about the same, confidence limits now asymmetric)
Confidence limits for composite forecasts Suppose you forecast X and Y separately, but you are really interested in X+Y or X Y. It is OK to add or multiply the corresponding point forecasts,, but can you add or multiply the corresponding confidence limits?? NO!!! If the errors in forecasting X and Y are independent (i.e., uncorrelated), there are ways to approximate the confidence limits.
Confidence limits for composite forecasts: the basic principle In the case of a sum of forecasts, the variances of the forecast errors are additive*. In the case of a product of forecasts, the variances of the percentage forecast errors are (approximately) additive.* ** These relations lead to square root of sum of squares formulas for the standard errors and confidence limits. *Assuming statistical independence of X and Y errors **Strictly speaking, variances are additive in logged units
Example of sum of forecasts Suppose the forecast and CI for X is 20 ± 3 while the forecast and CI for Y is 30 ± 4 Then the forecast and CI for X+Y is: (20+ 30) ± 3 2 + 4 2 = 50 ± 5
Example of product of forecasts Suppose the forecast and CI for X is 20 ± 3% while the forecast and CI for Y is 30 ± 4% Then the forecast and CI for X Y X Y is: (20 30) ± 3% 2 + 4% 2 = 600± 5% (Note that it is necessary to translate CI s s into percentage terms before applying the product formula)
Retail sales revisited Forecast & 50% CI for RetailxAuto/CPI in next period (March 02) is 125138 ± 1.80% Forecast & CI for CPI is 1.7775 ± 0.175% Forecast for RetailxAuto (i.e., the product) ) is: 2 2 (125138 1.7775) ± 1.80% + 0.175% = 222433 ± 1.81% Notice that because the % error in the CPI forecast is smaller by a factor of 1/10 (which becomes a factor of 1/100 when squared), it can essentially be ignored in this case.
Are errors independent? The actual correlation between errors for RetailxAuto/CPI and CPI is only -0.01: This report was obtained by saving the residuals of both models to the spreadsheet, then running the Describe/Numeric Data/Multiple-Variable procedure This correlation is not significantly different from zero, so the approximation for the CI s is OK Positive correlation would imply wider CI s, negative correlation would imply narrower CI s
Class 3 Recap Discussion of HW#1 Introduction to seasonal models Seasonal decomposition Seasonal adjustment on a spreadsheet Forecasting with seasonal adjustment Forecasting inflation Poor man s deflator Confidence intervals for composite forecasts
For next time Read handout on exponential smoothing Watch video clips #10-14 14 HW#2 is due a week from today