NATCOR. Forecast Evaluation. Forecasting with ARIMA models. Nikolaos Kourentzes
|
|
- Bertina Mills
- 5 years ago
- Views:
Transcription
1 NATCOR Forecast Evaluation Forecasting with ARIMA models Nikolaos Kourentzes
2 O u t l i n e 1. Bias measures 2. Accuracy measures 3. Evaluation schemes 4. Prediction intervals 5. Parameter selection 6. Method selection 2
3 F o r e c a s t E v a l u a t i o n Forecast errors Inaccurate results Loss (performance, financial etc...) Measuring the loss is important but often hard to track Forecasting error can be used as a proxy. Therefore is it important to track and evaluate forecast errors. Forecast evaluation = key activity in the forecasting process. It is at the core of: Important for Forecast Monitoring Essential for Method Selection and Parameterisation 3
4 F o r e c a s t E r r o r - D e f i n i t i o n Sales (A t ) Forecast (F t ) SKU A Sales e 4 e e Week e t A t F t e 51 Positive errors Negative errors Error Week Underforecast Overforecast 4
5 M e a s u r e s o f B i a s Instead of considering the complete vector of errors we can aggregate them using: 1 ME n MdE n t1 e median ( e t t ) Mean Error: Most common measure of forecast bias Median Error Measures of bias show whether we typically over- or under-forecast. Ideally this should be as close to zero as possible. Forecasting method A ME: Forecasting method B ME: Positive bias Under-forecasting Forecasting method C ME: Negative bias Over-forecasting Unbiased
6 M e a s u r e s o f B i a s Error Sales Mean Error Mean error = In this case we typically forecast more than what we should. This forecast will lead to biased decisions Error Sales Mean Error Mean error =.1 This forecast shows no preference, therefore it is useful for objective decision making Error Sales Mean Error Mean error = In this case we typically forecast less than what we should. This forecast will lead to biased decisions
7 A N o t e o n M e a n a n d M e d i a n E r r o r s It is well known that the mean is affected by outliers and asymmetric distributions more than the median. In the context of forecasting: Median insensitive to extremes (outliers), summarises better normal performance. Mean sensitive to extremes (outliers), useful when we are interested in them. Substantial differences between mean and median errors error distribution may have outliers or be asymmetric. Distribution of errors 2 18 Mean Error Median Error Frequency Outliers affect mean strongly Error 7
8 B i a s a n d M a g n i t u d e o f E r r o r s - A c c u r a c y Mean Error = does not tell us if we are accurate, merely whether we are biased. To overcome this we can calculate squared (e t2 ) or absolute errors ( e t ), which do not cancel out once aggregated. e t e t 2 e t e 1 e 2 e Sum Mean No error? 8
9 M e a s u r e s o f A c c u r a c y S c a l e D e p e n d e n t Some common errors that can be defined using these operators are: 1. Mean Squared Error (MSE) Sensitive to outliers (squares) Non-intuitive (units are squared) Scale dependent 2. Root Mean Squared Error (RMSE) As MSE but resulting units not in squares Scale dependent 3. Mean Absolute Error (MAE) Robust to outliers Scale dependent RMSE MSE MAE 1 n 1 n A i F i 1 n n i1 n A i F i i1 n i1 A i F i 2 2 Scale dependent errors can only be used to compare different methods on the same time series. If a time series is in cars errors are also in cars! Similar problems occur due to the scale. Should not be used for comparisons across different time series! Obviously we can define median versions of the above errors. 9
10 A N o t e o n A b s o l u t e a n d S q u a r e d E r r o r s Distribution of errors (e t ) Distribution of absolute errors ( e t ) Distribution of absolute errors (e t 2 ) Frequency Mean Median Frequency Mean Median Frequency Notice how extreme the outliers become Mean Median Error Absolute error Squared error Mean: units Median: units Mean: 5,179 units 2 Median: 14,22 units 2 Mean: 224. units Median : units Squared errors are sensitive to outliers, as they are increased disproportionally to smaller errors. On the other hand, absolute errors do not rescale the errors and the contribution of outliers is not exaggerated. 1
11 P e r c e n t a g e E r r o r s S c a l e I n d e p e n d e n t In order to compare across different time series we define a series of scale independent measures; neither the level or the units of the original time series are important. Percentage errors (PE) Expresses errors as a ratio to actual level Free of units Requires a meaningful zero ( o C is not a meaningful zero), so that the actuals do not become negative. PE t A t A t F t Based on the percentage errors we can define percentage bias and accuracy metrics. These will be scale and unit independent and therefore allow comparisons and aggregations between time series. 11
12 P e r c e n t a g e E r r o r s S c a l e I n d e p e n d e n t Some common errors that can be defined using these operators are: 1. Mean Absolute Percentage Error (MAPE) Scale independent Very intuitive (method is % wrong) MAPE Biased: Positive and negative errors do not count equally! Requires non-zero and positive denominator 1 n n i1 A i A i F i 2. Symmetric Mean Absolute Error (smape) If you see it, avoid it! Has too many several issues! There are also median versions of the absolute percentage errors. MAPE bias example Actual = 1 Forecast = 9 MAPE = 1 /1 = 1% Actual = 9 Forecast = 1 MAPE = -1 /9 = % 12
13 P e r c e n t a g e E r r o r E x a m p l e Period Actual t Forecast t e t t A t F t = A t - F t t t t Period Actual t Forecast t e t t A t F t = A t - F t t t t PE APE = e t /A t = PE 4.72% 4.72% -1.% 1.%.98%.98% PE APE = e t /A t = PE 4.72% 4.72% Infinite Infinite.98%.98% MAPE = 2.23% MAPE = Infinite! If there is a zero (or a value close to zero) MAPE becomes infinite (or extremely large) Medians will typically allow you to calculate MAPE as infinite errors are ignored! 13
14 E x a m p l e o f S c a l e I n d e p e n d e n t E r r o r s 8 6 Sales (A t ) Forecast (F t ) SKU A MAE = 134. computers MAPE = 179.7% Sales Week SKU C Sales (A t ) Forecast (F t ) The MAE of the first series is dwarfed from the second s, because the scale of the second series is in millions. MAE has also the issue of units. Sales Week MAE = 6,117,39 iron nails MAPE = 26.6% 14
15 R e l a t i v e E r r o r s In order to compare against a benchmark method we define a series of relative error measures. Relative errors (RE) Expresses errors as a ratio to the errors of another forecasting model, typically the naive. Free of units (scale independent) Directly compares forecasting methods RE t A t At Ft F Benchmark Geometric Mean Relative Absolute Error (GMRAE) Absolute form of relative errors Error < 1 method better than benchmark Error > 1 method worse than benchmark Error = 1 method as good as benchmark GMRAE n i1 A i A i F F i Naive 1 n To summarise across time series we use again a GM of the GMRAE for each series 15
16 R e l a t i v e S u m m a r y E r r o r s Another way to calculate GMRAE is to use mean of logarithms of RAEs. GMRAE n i1 A i A i F F i Naive 1 n exp 1 n n i1 log A i A i F F i Naive An alternative error metric is AvRelMAE (Average Relative MAE). This is calculated as follows: Av RelMAE n i1 MAE MAE Forecast Benchmark 1 n exp 1 n n i1 log MAE MAE Forecast Benchmark The idea is the following: 1. Calculate MAE for each series (i = 1,,n) 2. Calculate ratios with benchmark MAE 3. Average across different series with a GM (as we use ratios) More robust to calculate than GMRAE, but less sensitive to individual errors Same interpretation as GMRAE 16
17 E r r o r M e a s u r e R e m a r k s Can be calculated across any forecast horizon or aggregation of forecast horizons. Scale independent errors can be aggregated across time series as well. There is no best error measure! Depends on the data and question at hand. Error measures can help in assessing the bias, accuracy and robustness of a forecasting method, as well as its ranking against other methods. Different forecasting error measures may output different model rankings. 17
18 F o r e c a s t M o n i t o r i n g Let us assume that we run two different forecasts and tracked their performance at every period. The blue forecast is more reactive and therefore better. But if this SKU was forecasted every month automatically without any human intervention, could we identify the problem? Sales SES(.5) SES(.25) SKU B - SES Sales 3 Out-ofsample t+1 to t+6 forecast error MAE Very high errors should signal an alert for manual intervention SES(.5) Week SES(.25) Week The alert occurs very fast The poor forecast produces very high errors for prolonged periods 18
19 F o r e c a s t M o n i t o r i n g We can monitor automated forecasts by tracking their errors. This can be implemented unstructured or in a control chart approach. The errors can be used raw or smoothed. Periods with unexpectedly high errors MAE Period 19
20 O u t l i n e 1. Bias measures 2. Accuracy measures 3. Evaluation schemes 4. Prediction intervals 5. Parameter selection 6. Method selection 2
21 In- a n d O u t - of- S a m p l e The historical observations can be split into two subsets: In-sample: used for model building and parameterisation. Out-of-sample: used for model evaluation. This is not used in building the model and is not seen by the model. We use it to simulate true forecasts, instead of waiting for new unobserved values in order to evaluate the forecasting performance of alternative forecasting models In-sample Out-of-sample 55 Units Use to build 35 the model Month Forecast origin Use to evaluate the model. Note that forecast is multiple steps ahead 21
22 S t a t i c O r i g i n E v a l u a t i o n The simplest evaluation produce a single forecast in the out-of-sample subset. Let the forecast horizon be 12 months and the holdout (out-of-sample) 24 months: Units Month Forecast origin We have a forecast for t+1, t+2,..., t+12. There is a single forecast origin, month 6. Let us assume that we are interested to forecast accurately t We have only 1 measurement: low confidence in our accuracy measurement This evaluation scheme is called static origin evaluation. 22
23 S t a t i c O r i g i n E v a l u a t i o n Use to fit the model In-sample Use to evaluate the model Out-of-sample (Holdout) Origin Forecasts Limitations: 1. One forecast per lead time needs long track history 2. Forecast susceptible to corruption strange origins or targets may affect quality of forecasts 3. Averaging over different lead times corrupts summary error statistic 23
24 R o l l i n g O r i g i n E v a l u a t i o n A way to overcome these limitations is the Rolling Origin Evaluation scheme. In-sample Out-of-sample Roll origin Origin In-sample (increased) Out-of-sample Roll origin Origin In-sample (increased) Out-of-sample Origin We roll the forecast origin until there is not enough out-of-sample to use for evaluation.
25 R o l l i n g O r i g i n E v a l u a t i o n Holdout: 5 Holdout: 1 Forecast Lead Fixed Origin Rolling Origin Fixed Origin Rolling Origin Time Number of forecasts Number of forecasts Number of forecasts Number of forecasts t t t t t Rolling origin evaluation: 1. Provides more forecasts per origin 2. Overcomes limitations of fixed origin evaluation Provides more forecasting history per lead time for equal holdout sample Does not need to average over lead time Can overcome strange origins or targets 25
26 Alpha =.2 Alpha =.5 R o l l i n g O r i g i n E v a l u a t i o n Using the previous example a rolling origin evaluation would look like: In-sample Out-of-sample Black dots are forecast origins In-sample Out-of-sample Visualising the rolling origin forecasts makes it easier to appreciate the importance of smooth forecasts that filter noise
27 E v a l u a t i o n S c h e m e s S a m p l e S i z e Having enough sample for model building and evaluation is crucial. Lack of sample severely restricts the selection of alternative models, as many require abundance of data. Eventually on really short time series we can only apply naive and simple average models. Models that require parameterisation perform better when there is ample data. The more available data the better the estimation of the parameters. Consider issues with setting the gamma parameter on seasonal exponential smoothing models. The same is true for evaluating forecasts. With large sample many errors can be calculated and therefore higher confidence on the estimated figure. Sample size also affects our understanding of time series components. How many observations are required to identify a seasonal time series? 27
28 E v a l u a t i o n S c h e m e s S a m p l e S i z e Lets try to find the t+1 forecast error of a method using absolute errors... For 84 errors the MAE is 45.75, but let us assume that this is unknown. The 1 st error is: The 2 nd error is: The 3 rd error is: The 2 th error is: The 3 th error is: Mean: 65.96, are we confident? - Mean: 67.7, are we confident? - Mean: 77.83, are we confident? - Mean: 57.3, are we confident? - Mean: 49.65, are we confident? Absolute Error (AE) AE Cumulative MAE Final MAE MAE: Observation
29 E v a l u a t i o n S c h e m e R e m a r k s Forecasting accuracy of one model is meaningful only as a relative size to another model or benchmark. An error of 5% or 5% is non-informative without comparing it to benchmarks. Naive methods make simple and effective benchmarks. There are no set rules determining the size of the out-of-sample (or holdout), however: o o o o It should be at least as long as the forecast horizon Leave enough in-sample data for model building Provide enough forecasts of the forecast horizon of interest A simple heuristic is 8% in-sample 2% out-of-sample, but often is inappropriate. 29
30 O u t l i n e 1. Bias measures 2. Accuracy measures 3. Evaluation schemes 4. Prediction intervals 5. Parameter selection 6. Method selection 3
31 P r e d i c t i o n I n t e r v a l s Data SES We can use the error distribution of valid models to formulate prediction intervals of the forecasting methods Probability Error PDF e t =A t -F t % 95.5% 68.2% x
32 P r e d i c t i o n I n t e r v a l s Starting from the sample standard deviation of the errors: se h 1 n n i1 2 e e th, i th For an unbiased model the mean of errors is zero. Probability % 95.5% 68.2% se h 1 n n et h, i MSEh i1 The forecast is the expected value. Adding and subtracting to the forecast the quantity z α/2 s e h gives us the prediction intervals. The standard score for normal distributions (valid models) is easy to calculate. 2 PI h Ft h za / 2 s e h Prediction Interval z α/2 - score 5%.67 9% % % 2.58 Using prediction intervals we can visualise the confidence in our forecasts. 32
33 P r e d i c t i o n I n t e r v a l s SKU C SKU A 8% and 9% prediction intervals UK Android Market Share US Air Passengers Observe that the prediction intervals vary with the series, method and horizon. We have more confidence to forecasts with tight PIs. 33
34 P r e d i c t i o n I n t e r v a l s Calculating the PI formula can be complicated as it requires in-sample MSE of multiple forecast horizons (h): PI h Ft h za/ 2se Ft h z h a/ 2 MSE h This can be obtained by calculating the rolling origin in-sample MSE of the relevant forecast horizon. Alternatively it can be approximated using the following formula: se h h MSEt1 This is the square root of the horizon multiplied by the squared root of the 1-step ahead in-sample mean squared error. Note that there is substantial empirical evidence that this is a very rough approximation. 34
35 O u t l i n e 1. Bias measures 2. Accuracy measures 3. Evaluation schemes 4. Prediction intervals 5. Parameter selection 6. Method selection 35
36 P a r a m e t e r S e l e c t i o n We have seen for the exponential smoothing methods that we can select the smoothing parameters based on the theoretical properties of the methods and the characteristics of the time series: Low parameters imply long weighted averages and therefore robustness against outliers and increased noise. Higher parameters imply shorter weighted averages, reacting faster to new information and handling better breaks in the series. However it may be desirable to automate the parameter selection process. We can use insample error metrics for this purpose. 36
37 P a r a m e t e r S e l e c t i o n Sales (A t ) Forecast (F t ) SKU A - SES = Sales (A t ) Forecast (F t ) SKU A - SES = Sales 4 3 Sales Week Week This parameter is better, resulting in a better fit (lower error). MSE = 27,337 < MSE = 35,58 Based on this idea we can optimise model parameters (for any exponential smoothing method or generally). 37
38 P a r a m e t e r S e l e c t i o n : S E S e x a m p l e We can calculate the in-sample MSE for various values of alpha and identify the value that gives the lowest error MSE Minimum In-sample MSE α = Alpha This result is very close to the one we chose manually (.1). Obviously for more complex methods we can optimise several parameters simultaneously. 38
39 P a r a m e t e r S e l e c t i o n : S E S e x a m p l e The same principle can be applied to choose the initialisation level value as well. Now we vary both initialisation and smoothing parameter. 6 5 Sales (A t ) Forecast (F t ) SKU B - SES 4 Sales Week Level = 1697 Alpha =.3237 As the number of parameters (including initialisation values) increases optimisation becomes more time consuming and requires more data. 39
40 P a r a m e t e r S e l e c t i o n Will the optimal parameters be always the best? In fact no. There are many reasons for this: Optimisation is done in-sample. The correlation between in-sample error and out-of-sample has been shown to be low. Optimisation is (typically) done using t+1 MSE. In practice we forecast for longer horizons and our pragmatic cost functions are different. MSE by construction is very reactive to extreme errors, which may distort the error surface that we are search for the optimal values. Sample limitations as the number of parameters to optimise increases. Minimum error may not be the business objective. Companies may prefer consistency of forecasts across origins instead. Optimisation is very useful for automation, however human experts should override identified parameters if they violate theory or objectives. 4
41 P a r a m e t e r S e l e c t i o n T r y i t o u t Experiment with setting the alpha parameter: Do you agree with the optimal value? Does the in- and out-of-sample behave the same way? 41
42 P a r a m e t e r S e l e c t i o n R e m a r k s Optimisation for complex models is sensitive to the starting conditions (local optima). Different sets of initial values and parameters may give different results. Optimising on bias does not make sense, as positive and negative errors cancel out. Optimisation results may change depending on the error metric used. MSE is common, but other metrics may be useful. Initial level log(mse) Minimum MAE Minimum MSE Minimum MAPE Alpha Sales Error Alpha Level MSE MAE MAPE SKU B - SES Week MAE MSE MAPE 42
43 M e t h o d S e l e c t i o n We can use similar principles to select the appropriate forecasting method. There are two major approaches: Using information criteria (usable only within a family of methods, e.g. exponential smoothing). Using a validation holdout sample and rolling origin evaluation. These compliment manual selection, based on understanding the characteristics of a time series. 43
44 M e t h o d S e l e c t i o n W h y F i t E r r o r s D o N o t W o r k For selecting between methods we cannot use the in-sample errors as we did we parameter selection. This is due to the fact that more complex models will tend to have lower fit errors, even if their forecasts perform worse Sales Level EXSM Trend-Season EXSM SKU B Sales In-sample Holdout Week Level Exponential Smoothing In-sample MAE t+1 : Out-of-sample MAE t+1-t+6 : < < Trend-Seasonal Exponential Smoothing In-sample MAE t+1 : Out-of-sample MAE t+1-t+6 : 195. More complex models are more flexible and have higher potential to overfit compared to simpler models. 44
45 M e t h o d S e l e c t i o n I n f o r m a t i o n C r i t e r i a One approach is to penalise the fit of more complex model for the number of parameters they have (= complexity). We define as information criteria: IC ln( MSE) pq( n) ln(mse) is the logarithm of the 1-step ahead in-sample MSE. p is the number of parameters (including initial values). Q(n) is the penalty function. n is the in-sample size. Akaike Information Criterion (AIC) Uses the following penalty function: Bayesian Information Criterion (BIC) Uses the following penalty function: Q ( n) 2/ n Q ( n) ln( n) / n BIC penalises more larger models. For exponential smoothing no significant differences in performance. 45
46 I n f o r m a t i o n C r i t e r i a E x a m p l e Sales Level EXSM Trend-Season EXSM SKU B Sales In-sample Holdout Week Level Exponential Smoothing In-sample MSE t+1 : 812,56.27 AIC: BIC: Out-of-sample MAE t+1-t+6 : < < Trend-Seasonal Exponential Smoothing In-sample MSE t+1 : 413,584.9 AIC: BIC: Out-of-sample MAE t+1-t+6 : 195. Both information criteria give us the correct answer. 46
47 M e t h o d S e l e c t i o n H o l d o u t s a m p l e For the other approach we simply measure the error at a subset of the series that is not used for fitting the models. Level Exponential Smoothing In-sample MAE t+1 : Validation MAE t+1-t+6 : Out-of-sample MAE t+1-t+6 : Trend-Seasonal Exponential Smoothing In-sample MAE t+1 : Validation MAE t+1-t+6 : Out-of-sample MAE t+1-t+6 : 195. Again we get the correct answer! Validation set: Rolling origin forecasts Out-of-sample: Rolling origin forecasts 47
48 M e t h o d S e l e c t i o n Information Criteria Pros - Easy to calculate. Cons - Applicable only within a single family of methods. - Cannot always be aligned with the true cost function of the company. Holdout Set Pros - Universal, can be used to select between any methods. - Can be fully aligned with the true cost function Cons - Lose sample for the validation set. - If sample size is not adequate for a reasonable rolling origin evaluation then the results may not be reliable. - Computationally complex. 48
49 M e t h o d a n d P a r a m e t e r S e l e c t i o n R e m a r k s Forecast evaluation can help us to automate both method and parameter selection. There are several alternative options often they produce similar results Reliable fully automatic performance (use forecast monitoring) The key benefit of statistics is automation essential for modern business forecasting problems Experienced human experts can outperform automatic methods Understand structure of the time series Choose method and parameters appropriately 49
50 Thank you for your attention! Questions? Nikolaos Kourentzes Lancaster University Management School Lancaster Centre for Forecasting - Lancaster, LA1 4YX n.kourentzes@lancaster.ac.uk Forecasting blog: Full or partial reproduction of the slides is not permitted without author s consent. Please contact n.kourentzes@lancaster.ac.uk for more information.
Using Temporal Hierarchies to Predict Tourism Demand
Using Temporal Hierarchies to Predict Tourism Demand International Symposium on Forecasting 29 th June 2015 Nikolaos Kourentzes Lancaster University George Athanasopoulos Monash University n.kourentzes@lancaster.ac.uk
More informationAdvances in promotional modelling and analytics
Advances in promotional modelling and analytics High School of Economics St. Petersburg 25 May 2016 Nikolaos Kourentzes n.kourentzes@lancaster.ac.uk O u t l i n e 1. What is forecasting? 2. Forecasting,
More informationForecasting with R A practical workshop
Forecasting with R A practical workshop International Symposium on Forecasting 2016 19 th June 2016 Nikolaos Kourentzes nikolaos@kourentzes.com http://nikolaos.kourentzes.com Fotios Petropoulos fotpetr@gmail.com
More informationLecture 1: Introduction to Forecasting
NATCOR: Forecasting & Predictive Analytics Lecture 1: Introduction to Forecasting Professor John Boylan Lancaster Centre for Forecasting Department of Management Science Leading research centre in applied
More informationAnother Error Measure for Selection of the Best Forecasting Method: The Unbiased Absolute Percentage Error
Another Error Measure for Selection of the Best Forecasting Method: The Unbiased Absolute Percentage Error Fred Collopy The Weatherhead School of Management Case Western Reserve University Cleveland, Ohio
More informationManaging forecast performance or My adventures in Errorland. Steve Morlidge
Managing forecast performance or My adventures in Errorland. Steve Morlidge Steve Morlidge Unilever (1978 2006) roles include: Controller, Unilever Foods UK ($1 billion turnover) Leader, Dynamic Performance
More informationMeasuring the behaviour of experts on demand forecasting: a complex task
Measuring the behaviour of experts on demand forecasting: a complex task Nikolaos Kourentzes a,, Juan R. Trapero b, Ivan Svetunkov a a Lancaster University Management School Department of Management Science,
More informationHow Accurate is My Forecast?
How Accurate is My Forecast? Tao Hong, PhD Utilities Business Unit, SAS 15 May 2012 PLEASE STAND BY Today s event will begin at 11:00am EDT The audio portion of the presentation will be heard through your
More informationChapter 13: Forecasting
Chapter 13: Forecasting Assistant Prof. Abed Schokry Operations and Productions Management First Semester 2013-2014 Chapter 13: Learning Outcomes You should be able to: List the elements of a good forecast
More informationAutomatic Forecasting
Automatic Forecasting Summary The Automatic Forecasting procedure is designed to forecast future values of time series data. A time series consists of a set of sequential numeric data taken at equally
More informationIndustrial Engineering Prof. Inderdeep Singh Department of Mechanical & Industrial Engineering Indian Institute of Technology, Roorkee
Industrial Engineering Prof. Inderdeep Singh Department of Mechanical & Industrial Engineering Indian Institute of Technology, Roorkee Module - 04 Lecture - 05 Sales Forecasting - II A very warm welcome
More informationOperations Management
3-1 Forecasting Operations Management William J. Stevenson 8 th edition 3-2 Forecasting CHAPTER 3 Forecasting McGraw-Hill/Irwin Operations Management, Eighth Edition, by William J. Stevenson Copyright
More informationIntroduction to Statistical modeling: handout for Math 489/583
Introduction to Statistical modeling: handout for Math 489/583 Statistical modeling occurs when we are trying to model some data using statistical tools. From the start, we recognize that no model is perfect
More informationAUTO SALES FORECASTING FOR PRODUCTION PLANNING AT FORD
FCAS AUTO SALES FORECASTING FOR PRODUCTION PLANNING AT FORD Group - A10 Group Members: PGID Name of the Member 1. 61710956 Abhishek Gore 2. 61710521 Ajay Ballapale 3. 61710106 Bhushan Goyal 4. 61710397
More informationAn approach to make statistical forecasting of products with stationary/seasonal patterns
An approach to make statistical forecasting of products with stationary/seasonal patterns Carlos A. Castro-Zuluaga (ccastro@eafit.edu.co) Production Engineer Department, Universidad Eafit Medellin, Colombia
More informationDEPARTMENT OF QUANTITATIVE METHODS & INFORMATION SYSTEMS
DEPARTMENT OF QUANTITATIVE METHODS & INFORMATION SYSTEMS Moving Averages and Smoothing Methods ECON 504 Chapter 7 Fall 2013 Dr. Mohammad Zainal 2 This chapter will describe three simple approaches to forecasting
More informationImproved Holt Method for Irregular Time Series
WDS'08 Proceedings of Contributed Papers, Part I, 62 67, 2008. ISBN 978-80-7378-065-4 MATFYZPRESS Improved Holt Method for Irregular Time Series T. Hanzák Charles University, Faculty of Mathematics and
More informationGlossary. The ISI glossary of statistical terms provides definitions in a number of different languages:
Glossary The ISI glossary of statistical terms provides definitions in a number of different languages: http://isi.cbs.nl/glossary/index.htm Adjusted r 2 Adjusted R squared measures the proportion of the
More informationModified Holt s Linear Trend Method
Modified Holt s Linear Trend Method Guckan Yapar, Sedat Capar, Hanife Taylan Selamlar and Idil Yavuz Abstract Exponential smoothing models are simple, accurate and robust forecasting models and because
More informationImproving forecasting by estimating time series structural components across multiple frequencies
Improving forecasting by estimating time series structural components across multiple frequencies Nikolaos Kourentzes a,, Fotios Petropoulos a, Juan R. Trapero b a Lancaster University Management School
More informationA Joint Bayesian Forecasting Model of Judgment and Observed Data
Please cite this paper as: Davydenko, A., & Fildes, R. (2012). A joint Bayesian forecasting model of judgment and observed data (LUMS Working Paper 2012:4). Lancaster University: The Department of Management
More informationOld dog, new tricks: a modelling view of simple moving averages
Old dog, new tricks: a modelling view of simple moving averages Ivan Svetunkov a, Fotios Petropoulos b, a Centre for Marketing Analytics and Forecasting, Lancaster University Management School, UK b School
More informationWarwick Business School Forecasting System. Summary. Ana Galvao, Anthony Garratt and James Mitchell November, 2014
Warwick Business School Forecasting System Summary Ana Galvao, Anthony Garratt and James Mitchell November, 21 The main objective of the Warwick Business School Forecasting System is to provide competitive
More informationDecision 411: Class 3
Decision 411: Class 3 Discussion of HW#1 Introduction to seasonal models Seasonal decomposition Seasonal adjustment on a spreadsheet Forecasting with seasonal adjustment Forecasting inflation Poor man
More informationBUSI 460 Suggested Answers to Selected Review and Discussion Questions Lesson 7
BUSI 460 Suggested Answers to Selected Review and Discussion Questions Lesson 7 1. The definitions follow: (a) Time series: Time series data, also known as a data series, consists of observations on a
More informationExponential smoothing in the telecommunications data
Available online at www.sciencedirect.com International Journal of Forecasting 24 (2008) 170 174 www.elsevier.com/locate/ijforecast Exponential smoothing in the telecommunications data Everette S. Gardner
More informationSerie temporali: previsioni
A. A. 2016-2017 Serie temporali: previsioni prof. ing. Antonio Comi Department of Enterprise Engineering University of Rome Tor Vergata Bibliography Forecasting: principles and practice by Rob J Hyndman
More informationTracking Accuracy: An Essential Step to Improve Your Forecasting Process
Tracking Accuracy: An Essential Step to Improve Your Forecasting Process Presented by Eric Stellwagen President & Co-founder Business Forecast Systems, Inc. estellwagen@forecastpro.com Business Forecast
More informationDecision 411: Class 3
Decision 411: Class 3 Discussion of HW#1 Introduction to seasonal models Seasonal decomposition Seasonal adjustment on a spreadsheet Forecasting with seasonal adjustment Forecasting inflation Poor man
More informationAutomatic forecasting with a modified exponential smoothing state space framework
ISSN 1440-771X Department of Econometrics and Business Statistics http://www.buseco.monash.edu.au/depts/ebs/pubs/wpapers/ Automatic forecasting with a modified exponential smoothing state space framework
More informationWeek 5 Quantitative Analysis of Financial Markets Modeling and Forecasting Trend
Week 5 Quantitative Analysis of Financial Markets Modeling and Forecasting Trend Christopher Ting http://www.mysmu.edu/faculty/christophert/ Christopher Ting : christopherting@smu.edu.sg : 6828 0364 :
More informationChapter 5: Forecasting
1 Textbook: pp. 165-202 Chapter 5: Forecasting Every day, managers make decisions without knowing what will happen in the future 2 Learning Objectives After completing this chapter, students will be able
More informationDecision 411: Class 3
Decision 411: Class 3 Discussion of HW#1 Introduction to seasonal models Seasonal decomposition Seasonal adjustment on a spreadsheet Forecasting with seasonal adjustment Forecasting inflation Log transformation
More informationFour Basic Steps for Creating an Effective Demand Forecasting Process
Four Basic Steps for Creating an Effective Demand Forecasting Process Presented by Eric Stellwagen President & Cofounder Business Forecast Systems, Inc. estellwagen@forecastpro.com Business Forecast Systems,
More informationMEASURING FORECASTER PERFORMANCE IN A COLLABORATIVE SETTING WITH FIELD SALES, CUSTOMERS OR SUPPLIER PARTNERS
MEASURING FORECASTER PERFORMANCE IN A COLLABORATIVE SETTING WITH FIELD SALES, CUSTOMERS OR SUPPLIER PARTNERS Hans Levenbach, Ph.D. Preview: Measuring performance of forecasters is a complex task, especially
More informationObserving Dark Worlds (Final Report)
Observing Dark Worlds (Final Report) Bingrui Joel Li (0009) Abstract Dark matter is hypothesized to account for a large proportion of the universe s total mass. It does not emit or absorb light, making
More informationAssistant Prof. Abed Schokry. Operations and Productions Management. First Semester
Chapter 3 Forecasting Assistant Prof. Abed Schokry Operations and Productions Management First Semester 2010 2011 Chapter 3: Learning Outcomes You should be able to: List the elements of a good forecast
More informationForecasting Chapter 3
Forecasting Chapter 3 Introduction Current factors and conditions Past experience in a similar situation 2 Accounting. New product/process cost estimates, profit projections, cash management. Finance.
More informationMachine Learning Linear Regression. Prof. Matteo Matteucci
Machine Learning Linear Regression Prof. Matteo Matteucci Outline 2 o Simple Linear Regression Model Least Squares Fit Measures of Fit Inference in Regression o Multi Variate Regession Model Least Squares
More informationTime Series Forecasting Methods:
Corso di TRASPORTI E TERRITORIO Time Series Forecasting Methods: EXPONENTIAL SMOOTHING DOCENTI Agostino Nuzzolo (nuzzolo@ing.uniroma2.it) Antonio Comi (comi@ing.uniroma2.it) 1 Classificazione dei metodi
More informationTIME SERIES ANALYSIS AND FORECASTING USING THE STATISTICAL MODEL ARIMA
CHAPTER 6 TIME SERIES ANALYSIS AND FORECASTING USING THE STATISTICAL MODEL ARIMA 6.1. Introduction A time series is a sequence of observations ordered in time. A basic assumption in the time series analysis
More informationAnalytics for an Online Retailer: Demand Forecasting and Price Optimization
Analytics for an Online Retailer: Demand Forecasting and Price Optimization Kris Johnson Ferreira Technology and Operations Management Unit, Harvard Business School, kferreira@hbs.edu Bin Hong Alex Lee
More informationSimple robust averages of forecasts: Some empirical results
Available online at www.sciencedirect.com International Journal of Forecasting 24 (2008) 163 169 www.elsevier.com/locate/ijforecast Simple robust averages of forecasts: Some empirical results Victor Richmond
More informationComparing the Univariate Modeling Techniques, Box-Jenkins and Artificial Neural Network (ANN) for Measuring of Climate Index
Applied Mathematical Sciences, Vol. 8, 2014, no. 32, 1557-1568 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2014.4150 Comparing the Univariate Modeling Techniques, Box-Jenkins and Artificial
More informationLinear Regression 9/23/17. Simple linear regression. Advertising sales: Variance changes based on # of TVs. Advertising sales: Normal error?
Simple linear regression Linear Regression Nicole Beckage y " = β % + β ' x " + ε so y* " = β+ % + β+ ' x " Method to assess and evaluate the correlation between two (continuous) variables. The slope of
More informationChart types and when to use them
APPENDIX A Chart types and when to use them Pie chart Figure illustration of pie chart 2.3 % 4.5 % Browser Usage for April 2012 18.3 % 38.3 % Internet Explorer Firefox Chrome Safari Opera 35.8 % Pie chart
More informationAnother look at measures of forecast accuracy
International Journal of Forecasting 22 (2006) 679 688 www.elsevier.com/locate/ijforecast Another look at measures of forecast accuracy Rob J. Hyndman a, *, Anne B. Koehler b,1 a Department of Econometrics
More informationDiploma Part 2. Quantitative Methods. Examiner s Suggested Answers
Diploma Part Quantitative Methods Examiner s Suggested Answers Question 1 (a) The standard normal distribution has a symmetrical and bell-shaped graph with a mean of zero and a standard deviation equal
More informationThe SAB Medium Term Sales Forecasting System : From Data to Planning Information. Kenneth Carden SAB : Beer Division Planning
The SAB Medium Term Sales Forecasting System : From Data to Planning Information Kenneth Carden SAB : Beer Division Planning Planning in Beer Division F Operational planning = what, when, where & how F
More informationCHAPTER 4: DATASETS AND CRITERIA FOR ALGORITHM EVALUATION
CHAPTER 4: DATASETS AND CRITERIA FOR ALGORITHM EVALUATION 4.1 Overview This chapter contains the description about the data that is used in this research. In this research time series data is used. A time
More informationForecasting. Chapter Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
Forecasting Chapter 15 15-1 Chapter Topics Forecasting Components Time Series Methods Forecast Accuracy Time Series Forecasting Using Excel Time Series Forecasting Using QM for Windows Regression Methods
More informationWeek 2 Video 4. Metrics for Regressors
Week 2 Video 4 Metrics for Regressors Metrics for Regressors Linear Correlation MAE/RMSE Information Criteria Linear correlation (Pearson s correlation) r(a,b) = When A s value changes, does B change in
More informationChapter 7 Forecasting Demand
Chapter 7 Forecasting Demand Aims of the Chapter After reading this chapter you should be able to do the following: discuss the role of forecasting in inventory management; review different approaches
More informationForecasting: The First Step in Demand Planning
Forecasting: The First Step in Demand Planning Jayant Rajgopal, Ph.D., P.E. University of Pittsburgh Pittsburgh, PA 15261 In a supply chain context, forecasting is the estimation of future demand General
More informationForecasting Module 2. Learning Objectives. Trended Data. By Sue B. Schou Phone:
Forecasting Module 2 By Sue B. Schou Phone: 8-282-408 Email: schosue@isu.edu Learning Objectives Make forecast models using trend analysis in Minitab Make forecast models using Holt s exponential smoothing
More informationCP:
Adeng Pustikaningsih, M.Si. Dosen Jurusan Pendidikan Akuntansi Fakultas Ekonomi Universitas Negeri Yogyakarta CP: 08 222 180 1695 Email : adengpustikaningsih@uny.ac.id Operations Management Forecasting
More informationFORECASTING STANDARDS CHECKLIST
FORECASTING STANDARDS CHECKLIST An electronic version of this checklist is available on the Forecasting Principles Web site. PROBLEM 1. Setting Objectives 1.1. Describe decisions that might be affected
More information22/04/2014. Economic Research
22/04/2014 Economic Research Forecasting Models for Exchange Rate Tuesday, April 22, 2014 The science of prognostics has been going through a rapid and fruitful development in the past decades, with various
More informationinterval forecasting
Interval Forecasting Based on Chapter 7 of the Time Series Forecasting by Chatfield Econometric Forecasting, January 2008 Outline 1 2 3 4 5 Terminology Interval Forecasts Density Forecast Fan Chart Most
More informationFORECASTING COARSE RICE PRICES IN BANGLADESH
Progress. Agric. 22(1 & 2): 193 201, 2011 ISSN 1017-8139 FORECASTING COARSE RICE PRICES IN BANGLADESH M. F. Hassan*, M. A. Islam 1, M. F. Imam 2 and S. M. Sayem 3 Department of Agricultural Statistics,
More informationUnivariate ARIMA Models
Univariate ARIMA Models ARIMA Model Building Steps: Identification: Using graphs, statistics, ACFs and PACFs, transformations, etc. to achieve stationary and tentatively identify patterns and model components.
More informationChapter 8 - Forecasting
Chapter 8 - Forecasting Operations Management by R. Dan Reid & Nada R. Sanders 4th Edition Wiley 2010 Wiley 2010 1 Learning Objectives Identify Principles of Forecasting Explain the steps in the forecasting
More informationComplex exponential Smoothing. Ivan Svetunkov Nikolaos Kourentzes Robert Fildes
Complex exponential Smoothing Ivan Svetunkov Nikolaos Kourentzes Robert Fildes 18 March 2015 Introduction Exponential Smoothing methods performed very well in many competitions: M-Competitions in 1982
More informationForecasting Using Time Series Models
Forecasting Using Time Series Models Dr. J Katyayani 1, M Jahnavi 2 Pothugunta Krishna Prasad 3 1 Professor, Department of MBA, SPMVV, Tirupati, India 2 Assistant Professor, Koshys Institute of Management
More informationARIMA modeling to forecast area and production of rice in West Bengal
Journal of Crop and Weed, 9(2):26-31(2013) ARIMA modeling to forecast area and production of rice in West Bengal R. BISWAS AND B. BHATTACHARYYA Department of Agricultural Statistics Bidhan Chandra Krishi
More informationDemand Forecasting. for. Microsoft Dynamics 365 for Operations. User Guide. Release 7.1. April 2018
Demand Forecasting for Microsoft Dynamics 365 for Operations User Guide Release 7.1 April 2018 2018 Farsight Solutions Limited All Rights Reserved. Portions copyright Business Forecast Systems, Inc. This
More informationCopyright 2010 Pearson Education, Inc. Publishing as Prentice Hall.
13 Forecasting PowerPoint Slides by Jeff Heyl For Operations Management, 9e by Krajewski/Ritzman/Malhotra 2010 Pearson Education 13 1 Forecasting Forecasts are critical inputs to business plans, annual
More informationVisualization of distance measures implied by forecast evaluation criteria
Visualization of distance measures implied by forecast evaluation criteria Robert M. Kunst kunst@ihs.ac.at Institute for Advanced Studies Vienna and University of Vienna Presentation at RSS International
More informationResampling techniques for statistical modeling
Resampling techniques for statistical modeling Gianluca Bontempi Département d Informatique Boulevard de Triomphe - CP 212 http://www.ulb.ac.be/di Resampling techniques p.1/33 Beyond the empirical error
More informationA Multistage Modeling Strategy for Demand Forecasting
Paper SAS5260-2016 A Multistage Modeling Strategy for Demand Forecasting Pu Wang, Alex Chien, and Yue Li, SAS Institute Inc. ABSTRACT Although rapid development of information technologies in the past
More informationSolar irradiance forecasting for Chulalongkorn University location using time series models
Senior Project Proposal 2102490 Year 2016 Solar irradiance forecasting for Chulalongkorn University location using time series models Vichaya Layanun ID 5630550721 Advisor: Assist. Prof. Jitkomut Songsiri
More informationFVA Analysis and Forecastability
FVA Analysis and Forecastability Michael Gilliland, CFPIM Product Marketing Manager - Forecasting SAS About SAS World s largest private software company $2.43 billion revenue in 2010 50,000 customer sites
More informationForecasting. Dr. Richard Jerz rjerz.com
Forecasting Dr. Richard Jerz 1 1 Learning Objectives Describe why forecasts are used and list the elements of a good forecast. Outline the steps in the forecasting process. Describe at least three qualitative
More informationTime Series Forecasting Methods
Time Series Forecasting Methods Nate Derby Statis Pro Data Analytics Seattle, WA, USA Edmonton SAS Users Group, 11/13/09 Nate Derby Time Series Forecasting Methods 1 / 43 Outline Introduction 1 Introduction
More informationNew Statistical Methods That Improve on MLE and GLM Including for Reserve Modeling GARY G VENTER
New Statistical Methods That Improve on MLE and GLM Including for Reserve Modeling GARY G VENTER MLE Going the Way of the Buggy Whip Used to be gold standard of statistical estimation Minimum variance
More informationMachine Learning for OR & FE
Machine Learning for OR & FE Regression II: Regularization and Shrinkage Methods Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com
More informationRegression I: Mean Squared Error and Measuring Quality of Fit
Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving
More informationWhy is the field of statistics still an active one?
Why is the field of statistics still an active one? It s obvious that one needs statistics: to describe experimental data in a compact way, to compare datasets, to ask whether data are consistent with
More informationCHAPTER 18. Time Series Analysis and Forecasting
CHAPTER 18 Time Series Analysis and Forecasting CONTENTS STATISTICS IN PRACTICE: NEVADA OCCUPATIONAL HEALTH CLINIC 18.1 TIME SERIES PATTERNS Horizontal Pattern Trend Pattern Seasonal Pattern Trend and
More informationAustrian Inflation Rate
Austrian Inflation Rate Course of Econometric Forecasting Nadir Shahzad Virkun Tomas Sedliacik Goal and Data Selection Our goal is to find a relatively accurate procedure in order to forecast the Austrian
More informationGL Garrad Hassan Short term power forecasts for large offshore wind turbine arrays
GL Garrad Hassan Short term power forecasts for large offshore wind turbine arrays Require accurate wind (and hence power) forecasts for 4, 24 and 48 hours in the future for trading purposes. Receive 4
More informationForecasting. Operations Analysis and Improvement Spring
Forecasting Operations Analysis and Improvement 2015 Spring Dr. Tai-Yue Wang Industrial and Information Management Department National Cheng Kung University 1-2 Outline Introduction to Forecasting Subjective
More informationAvailable online Journal of Scientific and Engineering Research, 2015, 2(2): Research Article
Available online www.jsaer.com,, ():- Research Article ISSN: - CODEN(USA): JSERBR Measuring the Forecasting Accuracy for Masters Energy Oil and Gas Products Ezeliora Chukwuemeka D, Umeh Maryrose N, Mbabuike
More informationModelling Wind Farm Data and the Short Term Prediction of Wind Speeds
Modelling Wind Farm Data and the Short Term Prediction of Wind Speeds An Investigation into Wind Speed Data Sets Erin Mitchell Lancaster University 6th April 2011 Outline 1 Data Considerations Overview
More informationCHAPTER 4 VARIABILITY ANALYSES. Chapter 3 introduced the mode, median, and mean as tools for summarizing the
CHAPTER 4 VARIABILITY ANALYSES Chapter 3 introduced the mode, median, and mean as tools for summarizing the information provided in an distribution of data. Measures of central tendency are often useful
More informationGDP growth and inflation forecasting performance of Asian Development Outlook
and inflation forecasting performance of Asian Development Outlook Asian Development Outlook (ADO) has been the flagship publication of the Asian Development Bank (ADB) since 1989. Issued twice a year
More informationInput-variable Specification for Neural Networks An Analysis of Forecasting Low and High Time Series Frequency
Universität Hamburg Institut für Wirtschaftsinformatik Prof. Dr. D.B. Preßmar Input-variable Specification for Neural Networks An Analysis of Forecasting Low and High Time Series Frequency Dr. Sven F.
More informationPercentage errors can ruin your day. Roland Martin, Stephan Kolassa June 27, 2012
Percentage errors can ruin your day Roland Martin, Stephan Kolassa June 27, 2012 SAP The problem Forecast Accuracy measurement Used to determine the success of the forecast process Forecasting method,
More informationSTA 6104 Financial Time Series. Moving Averages and Exponential Smoothing
STA 6104 Financial Time Series Moving Averages and Exponential Smoothing Smoothing Our objective is to predict some future value Y n+k given a past history {Y 1, Y 2,..., Y n } of observations up to time
More information3. If a forecast is too high when compared to an actual outcome, will that forecast error be positive or negative?
1. Does a moving average forecast become more or less responsive to changes in a data series when more data points are included in the average? 2. Does an exponential smoothing forecast become more or
More informationForecasting in the presence of recent structural breaks
Forecasting in the presence of recent structural breaks Second International Conference in memory of Carlo Giannini Jana Eklund 1, George Kapetanios 1,2 and Simon Price 1,3 1 Bank of England, 2 Queen Mary
More informationRegression, Ridge Regression, Lasso
Regression, Ridge Regression, Lasso Fabio G. Cozman - fgcozman@usp.br October 2, 2018 A general definition Regression studies the relationship between a response variable Y and covariates X 1,..., X n.
More informationSegmenting electrical load time series for forecasting? An empirical evaluation of daily UK load patterns
Segmenting electrical load time series for forecasting? An empirical evaluation of daily UK load patterns Sven F. Crone and Nikolaos Kourentzes Abstract Forecasting future electricity load represents one
More informationModel Selection. Frank Wood. December 10, 2009
Model Selection Frank Wood December 10, 2009 Standard Linear Regression Recipe Identify the explanatory variables Decide the functional forms in which the explanatory variables can enter the model Decide
More informationQMT 3001 BUSINESS FORECASTING. Exploring Data Patterns & An Introduction to Forecasting Techniques. Aysun KAPUCUGİL-İKİZ, PhD.
1 QMT 3001 BUSINESS FORECASTING Exploring Data Patterns & An Introduction to Forecasting Techniques Aysun KAPUCUGİL-İKİZ, PhD. Forecasting 2 1 3 4 2 5 6 3 Time Series Data Patterns Horizontal (stationary)
More informationVariable Selection in Predictive Regressions
Variable Selection in Predictive Regressions Alessandro Stringhi Advanced Financial Econometrics III Winter/Spring 2018 Overview This chapter considers linear models for explaining a scalar variable when
More informationAutomatic Identification of Time Series Features for Rule-Based Forecasting
Marquette University e-publications@marquette Management Faculty Research and Publications Management, Department of 4-1-2001 Automatic Identification of Time Series Features for Rule-Based Forecasting
More informationDo we need Experts for Time Series Forecasting?
Do we need Experts for Time Series Forecasting? Christiane Lemke and Bogdan Gabrys Bournemouth University - School of Design, Engineering and Computing Poole House, Talbot Campus, Poole, BH12 5BB - United
More informationStatistics 262: Intermediate Biostatistics Model selection
Statistics 262: Intermediate Biostatistics Model selection Jonathan Taylor & Kristin Cobb Statistics 262: Intermediate Biostatistics p.1/?? Today s class Model selection. Strategies for model selection.
More informationForecasting exchange rate volatility using conditional variance models selected by information criteria
Forecasting exchange rate volatility using conditional variance models selected by information criteria Article Accepted Version Brooks, C. and Burke, S. (1998) Forecasting exchange rate volatility using
More information