NATCOR: Forecasing & Predicive Analyics Lecure 3: Exponenial Smoohing John Boylan Lancaser Cenre for Forecasing Deparmen of Managemen Science
Mehods and Models Forecasing Mehod A (numerical) procedure for generaing a forecas. eg Take he average of all observaions up o (and including) ime, as a forecas for ime 1. 1 yˆ 1 = yi Forecasing Model A saisical descripion of he daa generaing process. eg All observaions are cenred around an unchanging mean (μ) wih a normally disribued i.i.d. noise erm ( ε ~ N(0, V )) wih zero mean and consan variance (V). = µ ε y i= 1 Slide 2 NATCOR Exponenial Smoohing
Link beween Models and Mehods Heurisic Mehods (No Link) These are mehods ha have been designed wihou reference o saisical models and have no link o such models. eg Simple Moving Averages (see laer slides). Model-Based Mehods (Linked) These are mehods which do link o an explici saisical model and give he bes forecas if he model holds. eg Simple Exponenial Smoohing (see laer slides). eg Average of all observaions links o he model on he previous slide. y = µ ε Slide 3 NATCOR Exponenial Smoohing
Arihmeic Mean yˆ 1 = 1 i= 1 y i Gives equal weigh o all observaions Has longes possible memory. Reduces noise, as he random flucuaions end o cancel ou. The more daa is available, he longer he average, and he beer he esimaion of he mean level. If he model y = µ ε has held in he pas, and coninues o hold over he forecas horizon, hen he Arihmeic Mean is he bes forecas for ime 1. Wha forecas should be used for ime 2? Slide 4 NATCOR Exponenial Smoohing
Forecasing wih he Arihmeic Mean y ˆ = y 2 1 1 y ˆ3 2 = ( y1 y2) / 2 650 1 sep-ahead forecass: yˆ yˆ y ˆ 2 1, 3 2,..., 84 83 1-12 sep-ahead f/cass: yˆ yˆ y ˆ 85 84, 86 84,..., 96 84 600 550 Unis 500 450 400 350 10 20 30 40 50 60 70 80 90 Monh Forecas becomes more sable as ime progresses If model holds, forecas accuracy depends on level of noise in he model error erm Slide 5 NATCOR Exponenial Smoohing
Arihmeic Mean and Ouliers 600 500 Acuals Arihmeic Mean Oulier 400 300 200 100 10 20 30 40 50 60 70 80 90 100 110 The Arihmeic Mean becomes more robus o ouliers as he lengh of hisory grows. The weigh given o he oulier is only 1/, where is he lengh of he hisory used in calculaing he mean. Slide 6 NATCOR Exponenial Smoohing
Arihmeic Mean and Level Shifs 350 300 Acuals Arihmeic Mean 250 200 150 100 50 10 20 30 40 50 60 70 80 90 100 110 Level Shif occurs here The Arihmeic Mean is poor a handling level shifs. The mehod has a long memory. I canno forge he previous level and adus o he new level wihin a reasonable period of ime. Slide 7 NATCOR Exponenial Smoohing
Random Walk Model y = y 1 ε Mean level no longer consan (see graph) The nex noise erm ( ε 1 ) is no forecasable a ime Bes forecas of is o use he laes observaion ( ) y 1 y Slide 8 NATCOR Exponenial Smoohing
Naïve Forecas y ˆ = 1 y 650 600 550 Unis 500 450 400 350 10 20 30 40 50 60 70 80 90 Monh Naïve does no filer he noise - i copies he noise. Arihmeic Mean good a filering noise bu unresponsive o level shifs. The Naïve mehod is he opposie. Slide 9 NATCOR Exponenial Smoohing
Alernaive Approach: Simple Moving Averages Gives equal weigh o all of he las N observaions in he average: yˆ 1 = 1 N y i i= N 1 Memory depends on lengh of Simple Moving Average Unlike Arihmeic Mean and Naïve mehods, he Simple Moving Average has a parameer (N) ha needs o be deermined. Higher N values filer noise beer bu respond more slowly o level shifs. Mehod is no model-based bu may sill perform more accuraely han some model-based mehods (eg Naïve). Slide 10 NATCOR Exponenial Smoohing
Difference beween Simple and Cenred Moving Average Simple Moving Average Simple Moving Average (SMA) of lengh 3 akes he average of he firs hree observaions as a forecas for he fourh period. Cenred Moving Average (CMA) of lengh 3 akes he average of he firs hree observaions as an esimae of he underlying model a he second period. Slide 11 NATCOR Exponenial Smoohing
Effec of Lengh of SMA 650 600 550 Unis 500 450 400 350 Acuals SMA(6) SMA(12) SMA(24) 10 20 30 40 50 60 70 80 90 Monh Differen lenghs of SMA may produce quie differen forecass. Bes choice of lengh depends on wheher i is more imporan o filer noise or respond o level shifs. Slide 12 NATCOR Exponenial Smoohing
SMA and Ouliers 550 500 450 400 Acuals SMA(6) SMA(12) SMA(24) 350 300 250 200 150 100 10 20 30 40 50 60 70 80 90 100 110 Robusness of SMA o ouliers depends on lengh of SMA The longer he SMA, he more robus is he forecas o oulying observaions. Slide 13 NATCOR Exponenial Smoohing
SMA and Level Shifs 350 300 Acuals MA(6) MA(12) MA(24) 250 200 150 100 50 10 20 30 40 50 60 70 80 90 100 110 Adapaion of SMA o level shifs depends on lengh of SMA I will ake N periods for an SMA o fully adap o a new level (where N is he lengh of he SMA). Slide 14 NATCOR Exponenial Smoohing
Choice of Lengh (Order) of SMA Bes lengh of SMA no known in advance Times series graph may give some clues bu canno deermine bes lengh of SMA from his alone. Need o compare accuracy of SMA using differen lenghs. We experimen on pas daa, bu only using daa ha would have been available a he ime o calculae our forecass. Issues o resolve 1. Wha error measure? 2. How many seps-ahead? 3. Over wha ime period? Slide 15 NATCOR Exponenial Smoohing
Error Measures (h-sep-ahead forecass) Slide 16 Mean Squared Error (MSE) Mean Absolue Error (MAE) = = = = 1 0 1 0 2 2 ) ˆ ( 1 1 m m h h h y y m e m MSE = = = = 1 0 1 0 ˆ 1 1 m m h h h y y m e m MAE NATCOR Exponenial Smoohing Mean Absolue Percenage Error (MAPE) = = = = 1 0 1 0 ˆ 100 100 m m h h h h h y y y m y e m MAPE
1. Choice of Error Measure o deermine lengh of SMA Mos common choice is MSE. MSE is he error measure used in imes series heory o link models o mehods which are opimal (Minimum Mean Square Error, MMSE) for ha model. This is wha was mean by bes forecas in earlier slides. MSE also links o he AIC measure for model selecion (discussed laer). However, resuls can be sensiive o oulying observaions. Slide 17 NATCOR Exponenial Smoohing
2. Choice of Forecas Horizon (h) o deermine lengh of SMA Mos common choice is one-sep-ahead. If we are only ineresed in (say) 3-sep-ahead errors, hen we may minimise MSE for 3-sep-ahead forecass. Ofen, we are ineresed in 1-sep, 2-sep and 3-sepahead errors (say). Then minimising MSE for 1-sep-ahead forecass sands in for he oher wo horizons. Alernaive approaches, aking ino accoun all he relevan horizons, are currenly being researched by he Lancaser Cenre for Forecasing. Slide 18 NATCOR Exponenial Smoohing
3. Choice of Time Period over which o deermine lengh of SMA 12000 10000 US expors of upper and lining leaher In-sample Ou-of-sample Unis 8000 6000 4000 Daa In-sample forecas Ou-of-sample forecas Forecas origin 2000 Jan77 Jan79 Jan81 Jan83 Jan85 Jan87 Monh Divide hisory ino in-sample (raining se) and ou-ofsample (es se). Use in-sample o deermine lengh of SMA Use ou-of-sample o compare SMA wih oher mehods Slide 19 NATCOR Exponenial Smoohing
Example Series 140.00 130.00 120.00 110.00 100.00 90.00 80.00 70.00 60.00 Jan 2012 Apr 2012 Jul 2012 Oc 2012 Medium Noise Jan 2013 Apr 2013 Jul 2013 Oc 2013 Jan 2014 Apr 2014 Jul 2014 Oc 2014 Jan 2015 Apr 2015 Jul 2015 Oc 2015 260.00 240.00 220.00 200.00 180.00 160.00 140.00 120.00 100.00 80.00 Medium Noise wih Level Shif Jan 2012 Apr 2012 Jul 2012 Oc 2012 Jan 2013 Apr 2013 Jul 2013 Oc 2013 Jan 2014 Apr 2014 Jul 2014 Oc 2014 Jan 2015 Apr 2015 Jul 2015 Oc 2015 Open Exponenial Smoohing Exercise spreadshee a firs ab (Daa Visualisaion) for hese series in Columns A and C. Two addiional series High Noise, and High Noise wih Level Shif are in Columns B and D. Slide 20 NATCOR Exponenial Smoohing
Fixed Forecass and Rolling Forecass in Ou-of-Sample 12000 10000 US expors of upper and lining leaher In-sample Ou-of-sample Unis 8000 6000 4000 Daa In-sample forecas Ou-of-sample forecas Forecas origin 2000 Jan77 Jan79 Jan81 Jan83 Jan85 Jan87 Monh Graph shows fixed forecass, made a he Forecas Origin (ie one 1-sep-ahead f/cas, one 2-sep-ahead f/cas ec). Rolling forecass are made a he Origin, hen a he Origin plus one period, Origin plus wo periods ec. Slide 21 NATCOR Exponenial Smoohing
Fixed Forecass Spli beween In-Sample and Ou-of-Sample Accuracy will be assessed for all forecas horizons ou-ofsample, wih each f/cas made a he Forecas Origin. So, ou-of-sample lengh should be se o be equal o he longes forecas horizon. Rolling Forecass Trade off beween: 1. Longer in-sample lenghs allow more accurae assessmen of he opimal parameer (lengh of SMA). 2. Longer ou-of-sample lenghs allow for more accurae comparisons of differen mehods if using Rolling Forecass. Slide 22 NATCOR Exponenial Smoohing
Daa Spliing in EXCEL Daa Spliing Exercise Open spreadshee a 2nd ab (2. Daa Spliing) Experimen wih differen In-sample sizes (Cell K2, or use he slider bar below) for boh: Medium Noise Medium Noise wih Level Shif. Wha effec would changing he In-sample size have on esimaion of lengh of SMA in he Training Se and evaluaion of forecas accuracy in he Tes Se? Slide 23 NATCOR Exponenial Smoohing
Simple Exponenial Smoohing (SES) Suppose daa does no have seasonaliy or sysemaic rend Daa may have ouliers and/or level shifs. Exponenial Smoohing aduss he las forecas by a fracion (α) of he las forecas error: = yˆ 1 1 αe Example Previous Forecas = 100 Previous Acual = 90 Previous Error = -10 Smoohing Consan (α) = 0.2 New Forecas = 100 (0.2 x (-10)) = 98 yˆ Slide 24 NATCOR Exponenial Smoohing
SES: Error Correcion & Sandard Forms Error Correcion Form yˆ = yˆ 1 1 αe e = y ˆ y 1 Sandard Form Subsiue for he error expression in Error Correcion Form: yˆ yˆ ˆ ˆ 1 = y 1 αy αy 1 ˆ 1 = αy (1 α) y 1 This is a weighed average of he las acual and las forecas. Slide 25 MSCI 523 Exponenial Smoohing
Calculaion of SES Period Acual SES(0.3) Sqd Error SES(0.7) Sqd Error 1 90 2 85 90.0 25.0 90.0 25.0 3 83 88.5 30.3 86.5 12.3 4 92 86.9 26.5 84.1 63.2 5 98 88.4 92.3 89.6 70.3 6 81 91.3 105.6 95.5 209.8 7 94 88.2 33.7 85.3 74.9 8 150 89.9 3607.7 91.4 3433.5 9 86 108.0 482.0 132.4 2154.9 10 90 101.4 129.2 99.9 98.5 11 104 98.0 93.0 12 96 98.0 93.0 Overall MSE 563.4 764.7 Iniialise Forecas in period 2 by using Naïve mehod. Can hen opimise α (0 α 1). Alernaively, can opimise boh Iniial Forecas and α. Slide 26 NATCOR Exponenial Smoohing
SES in EXCEL SES Exercise Make sure you have he Solver Add-In (File, Opions, Add-Ins, Solver Add-In, OK) Open spreadshee a 6h ab: 6. Exponenial Smoohing) Selec Medium Noise wih Level Shif a 2 nd ab and hen reurn o 6 h ab. Inpu 24 o Cell X3 (In Sample Size). Iniialise forecas (naïve) in Cell C3 Calculae Training Se 1-sep-ahead forecass (C4:C26) Noe ha Tes Se forecass are all he same as C26. Experimen wih differen alpha values (Cell P3) Opimise alpha, and check Tes Se accuracy Slide 27 NATCOR Exponenial Smoohing
How SES addresses Noise Low smoohing consans (alpha values) filer noise. High smoohing consans have lile filering effec. BUT: high smoohing consans reac more quickly o level shifs. Slide 28 NATCOR Exponenial Smoohing
SES and Trended Series Alpha = 0.2 Alpha = 0.7 UK Gross Domesic Produc: chained volume measures 1400000 UK Gross Domesic Produc: chained volume measures 1400000 1200000 1200000 1000000 1000000 GDP 800000 GDP 800000 600000 600000 400000 400000 200000 1948 1958 1968 1978 1988 1998 2008 2018 Year 200000 1948 1958 1968 1978 1988 1998 2008 2018 Year Wih a low alpha, SES does no keep up wih rend and produces a poor forecas. Wih a high alpha, SES keeps up beer, bu is no filering he noise well and produces a forecas ha could be improved. Slide 29 NATCOR Exponenial Smoohing
SES and Seasonal Series Alpha = 0.2 Alpha = 0.7 110000 UK Hourly Elecriciy Demand 110000 UK Hourly Elecriciy Demand 100000 100000 90000 90000 Demand 80000 70000 Demand 80000 70000 60000 60000 50000 50000 40000 10/26/0810/27/08 10/28/0810/29/08 10/30/0810/31/08 10/26/08 Day 40000 10/26/0810/27/08 10/28/0810/29/08 10/30/0810/31/08 10/26/08 Day Wih a low alpha, seasonaliy is no capured. Wih a high alpha, he noise is no smoohed AND he seasonal paern is ou by one period. In boh cases, he forecass are poor. Slide 30 NATCOR Exponenial Smoohing
Is SES a Model-Based Mehod? I is someimes saed ha SES is an ad hoc or heurisic mehod, lacking a model-based foundaion. This is wrong! I is rue ha when SES was firs proposed, he mehod lacked a model foundaion. Since hen, wo model forms have been found o underpin SES: ARIMA(0,1,1) Model Sae Space Local Level Model Model formulaions become useful when looking a a whole family of Exponenial Smoohing models (including rend and seasonaliy). Slide 31 NATCOR Exponenial Smoohing
Summary Arihmeic Mean robus o ouliers bu very slow o respond o level shifs. Naïve responds immediaely o level shifs bu does no filer noise. Simple Moving Average (SMA) may be a good compromise bu is no par of a wider family of model-based mehods. Simple Exponenial Smoohing (SES) allows suiable weighs o be idenified for pas daa and is par of a wider family of model-based mehods. Slide 32 NATCOR Exponenial Smoohing