Using Forecast Errors for Business Optimisation

Transcription

1 Using Forecast Errors for Business Optimisation A guide for Project Managers David Evans BEng (Hons), GDipAppFin (Finsia) [e] evansd246@gmail.com

2 Disclaimer This publication contains general information only and the author is not, by means of this publication, rendering accounting, business, financial, investment, legal, tax, or other professional advice or services. This publication is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a qualified professional advisor. The author shall not be responsible for any loss sustained by any person who relies on this publication. Copyright All rights reserved. 2/57

3 Outline Exploring uncertainty and forecasting Quantifying and interpreting uncertainty in business Practical applications in the real world of business 3/57

4 Objectives of this presentation Change your perspective on forecasting Give you practical tools to take away and use 4/57

5 Exploring forecast errors and uncertainty 5/57

6 Uncertainty is a fact of life Forecasts are crucial for business planning, but they will always carry uncertainty Predictions using point estimates are perilous Business is a game of odds 6/57

7 Statistics versus judgement Statistical forecasts are generally superior to judgemental forecasts... But in many situations, judgemental forecasts are the only practical approach 7/57

8 Judgmental biases don t make things any easier though Conservatism Inconsistency Availability Selective perception Overconfidence Recency Illusory correlations Anchoring Attribution 8/57

9 Overconfidence is a major factor in judgmental bias It s important to consider uncertainty when making estimates and forecasts This is best explained with an example... 9/57

10 How many vehicles do you estimate are shown below? (Provide a range, not a point estimate) 10/57

11 There are actually 347 Even with a range of values most people didn t come close Even with things right in front of them! Narrow estimate ranges suggest overconfidence 11/57

12 Here are the results A B C D E F G H I Forecaster 12/57

13 Multi-billion-dollar guesses If people can t accurately predict the number of vehicles drawn on a slide... How can people expect to predict vehicles that will use a new billion-dollar tunnel or tollway? 13/57

14 Examining mistakes is important Forecast track records are usually ignored however Think of an infrastructure construction project that has failed spectacularly because of poor forecasts Imagine if people seriously started analysing forecasting track records of individuals! 14/57

15 Don t stress! It s perfectly normal to have forecast errors Performance isn t hitting the bullseye at every shot. That is a circus act. Peter Drucker 15/57

16 The issue of acting under uncertainty is always present Economists predicting recoveries Supermarket managers ordering stock Project managers estimating timelines (the list goes on) 16/57

17 Quantifying & interpreting forecast errors 17/57

18 A closer look at forecast errors Start by comparing actual figures with forecasts Beware: absolute figures are deceptive! Adding 5 to 1000 means 0.5% more; but adding 5 to 30 means 16.7% more 18/57

19 Relative forecast error is better than absolute forecast error Forecast Error Percentage is defined as FEP = (Actual Forecast)/(Actual) Another way uses ratios rather than percentages Forecast Error Ratio is defined as FER = (Actual)/(Forecast) 19/57

20 A quick refresher on some statistics terms The mean is commonly referred to as the average The standard deviation is a measure of variability that is expressed in the same units as the mean 20/57

21 Forecast errors are random variables Statistical tools can be applied to understand and interpret them further Random variables are everywhere in business Probability distributions are used to describe them 21/57

22 DIY forecast error distribution Think of it as a histogram of errors As a general rule use at least 30 observations 22/57

23 Example: Happy-Xpress Tollway Set to revolutionise the quality of life for the citizens of Happyville City with its 3km of tunnels Mike is Project Manager and had to forecast construction activity and budgets Each activity on his Gantt chart was a forecast 23/57

24 Frequency Sometimes he underestimated; and sometimes he overestimated His forecast error histogram begins to take shape -8% -6% -4% -2% 0% +2% +4% +6% +8% Forecast Error 24/57

25 Frequency Histogram grows to approximate a bell curve (Normal Distribution) Empirical data used to quantify uncertainty -8% -6% -4% -2% 0% +2% +4% +6% +8% Forecast Error 25/57

26 Frequency Lack of symmetry or bell shape indicates problems Skewing shows particular bias and inconsistency -8% -6% -4% -2% 0% +2% +4% +6% +8% Forecast Error 26/57

27 Frequency Unbiased forecasters yield a curve centred on 0% error Nice bell shape, but shows habit of overestimating -8% -6% -4% -2% 0% +2% +4% +6% +8% Forecast Error 27/57

28 Precision is not the same as accuracy Accuracy is about hitting the bullseye Precision is about constantly hitting the same spot 28/57

29 Frequency John is an accurate forecaster; however Sally is precise Fix her inaccuracy and she will be superior to John Sally John -8% -6% -4% -2% 0% +2% +4% +6% +8% Forecast Error 29/57

30 Summarising Mike s activity forecasting performance Normally distributed Average FER is 1.14 meaning on average, the actual duration is 14% more than what he forecasts Standard deviation of his FER is and this will be used to quantify uncertainty of future forecasts 30/57

31 Practical applications in the world of business 31/57

32 Mike s current project is the Happy-Rapid Roadway A $440 million infrastructure project to complete an east-west arterial for Happyville Funded by both tax-payers money and private investors 32/57

33 Mike s future forecasts can now be adjusted appropriately He forecasts surveying activities will take 37 days But since we know his forecast error statistics, we can analyse probabilities 33/57

34 Underestimating seems to be his habit, so we adjust upwards He says surveying will take 37 days... But on average things take 14% longer so expect the task to take 1.14 x 37 days = days (round to 43 days) 34/57

35 The FER standard deviation quantifies the uncertainty His FER standard deviation is x 37 days = 7.62 days (round to 8 days) For normally distributed random variables, there s a 68% chance of being ±1 standard deviations from the mean 35/57

36 Probability His FER statistics can be used to build a probability distribution Area under curve represents 100% of possibilities Lead time (days) 36/57

37 What s the chance of the task taking less than 40 days to finish? It s the proportion of curve area falling below 40 Use Excel to answer this quickly and easily Enter =normdist(40,43,8,1) into any cell Number of days in question Expected lead time is 43 days Standard deviation is 8 days Flag set to 1 for cumulative Result is meaning an ~35.4% chance 37/57

38 Probability There s a 35.4% chance the task will take less than 40 days 35.4% of the distribution area lies below 40 days 35.4% Lead time (days) 38/57

39 This question can also be asked in reverse There s a 75% chance the lead time will be less than what duration? Enter =norminv(0.75,43,8) into any cell Probability being sought Expected lead time is 43 days Standard deviation is 8 days The result is or 48 days 39/57

40 Probability That means there s a 25% chance it will take longer than 48 days 75% 25% Lead time (days) 40/57

41 This technique can be applied to more than just lead times Each activity also has an associated forecast cost The viability of the Happy-Rapid Roadway investment also hinges on forecast traffic levels 41/57

42 Traffic forecasts are the backbone of the Happy-Rapid business case But what s the uncertainty associated with the traffic forecasts? Point estimates simply aren t good enough 42/57

43 Use the odds of various traffic levels to analyse profit scenarios For example, if the forecast is 13,000 vehicles per day but 8,500 per day are needed to break even What s the probability of having only 8,500 vehicles per day using the roadway? Use the forecaster s error statistics to estimate 43/57

44 Tracking forecasting performance If something can t be measured, it can t be managed The statistical tools presented earlier can be used to measure forecasting accuracy and precision 44/57

45 There are a number of ways to track forecast error statistics Numerical descriptive measures (means, medians, and standard deviations) Graphical descriptive measures (histograms, probability distributions) 45/57

46 Although picture speaks a thousand words So we ll use graphical descriptive measures 46/57

47 Forecast Error Percentage Looking at Mike s last 5 projects, it s clear he underestimates 30% 20% Maximum error +1 standard deviation 10% 0% -10% Average FEP 1 standard deviation Minimum error -20% -30% A B C D E Project 47/57

48 Forecast Error Percentage With effort applied, he begins to become more precise... 30% 20% 10% Note the reduction in dispersion 0% -10% -20% -30% F G H I J Project 48/57

49 Forecast Error Percentage And then eventually more accurate too, with less bias 30% 20% 10% 0% Note the average FEP is closer to zero -10% -20% -30% K L M N O Project 49/57

50 It won t guarantee success; but it s better than doing nothing! Forecasting mistakes should be discussed in the open as a learning opportunity It s as simple as keeping track of actual vs. forecast 50/57

51 Forecast Error Percentage It s also great for comparing project managers 30% 20% 10% Note the skewing 0% -10% -20% -30% Apparently Eric lacks precision but is accurate Tim is precise but tends to overestimate Kate is biased and underestimates Eric Tim Kate Project Manager 51/57

52 Lessons on uncertainty It s important to remember that forecasts are essentially nothing but guesses Instead of thinking in terms of black and white, we need to think in terms of grey and fuzzy 52/57

53 Lessons on forecasting Forecasts with point estimates are not enough! Need standard deviation to understand uncertainty Examining the odds of various scenarios is critical Don t be afraid! Forecast errors must be discussed 53/57

54 Lessons on improving forecasts It all begins by recording actual and forecast figures; this data forms the cornerstone Look for raw data on your track record and start improving things today. What s stopping you? These techniques are valid for all industries 54/57

55 Conclusion Life is uncertain but that s not an excuse! Business is a game of odds; some more than others Turn the odds in your favour with statistical tools 55/57

56 56/57 Questions?

57 Further reading /57