The Real Mystery of Demand Planning

Transcription

1 The Real Mystery of Demand Planning Jeff Metersky Chainalytics

2 Contents 1. Why Benchmark Demand Planning 2. Challenges with Existing Approaches 3. Introduction to Sales & Operations Variability Consortium 4. Influencers of Forecast Accuracy and Bias 5. Forecast Accuracy and Bias Segmentation Framework 6. Survey Results: Linking Policies, Approaches & Techniques to Performance 7. Predictors of Forecast Accuracy a) Profiles of Top vs. Bottom Performers b) Item Portfolio, Network & Demand Characteristics c) Forecasting Policies, Approaches & Techniques 1

3 Goals of Benchmarking Demand Planning How do we prioritize improvement opportunities? What are the underlying drivers of error, such as product portfolio, customer order patterns, economic cycles, seasonality, new product launches, etc.? What is causing our challenges in forecasting? Is our forecast accuracy and bias reasonable compared to competitors and peer companies using common metrics? How well do we forecast, relative to that inherent uncertainty? Are we better at forecasting some types of products than others? Does the way we compute error distort comparisons? What is our underlying demand uncertainty? 2

4 Survey-Based Benchmarking is not a Solid Foundation for Understanding your Effectiveness as a Forecaster A straight comparison with benchmark average MAPE is a reasonable indicator of forecast goodness, but not a definitive one because the answer really depends on your forecast environment and getting answers to some basic questions about it. Dr. Larry Lapide, Director, Demand Management, MIT Center for Transportation & Logistics Can published figures on sales forecasting accuracy serve as benchmarks? My analysis indicates that the survey results suffer from multiple sources of incomparability in the data on which they are based. These include differences in industry and product, in spatial and temporal granularity, in forecast horizon, in metric, in the forecast process, and in the business model. Stephan Kolassa, Vice President, Corporate Research at SAF AG 3

5 Change in FCA Due to Missing Forecasts or Missing Shipment Adjustments Existing Benchmarks Lack of Common Measures Forecasted Orders 37% Forecasted Orders (Adj. for Shorts) 4% Sales Order Received 11% Actual Shipments 48% -4.0% Adjusted for Missed Activity -3.0% -2.0% -1.0% 0.0% Members 4

6 Sales & Operations Variability Consortium (S&OVC) Industry: Consumer Product Goods, Food & Beverage Geography: U.S. Customer Demand (some Canada) Members: 27 Participants Item-Locations: 90,000+ Global Sales: $391B+ Home Care 21% Others 11% Personal Care 36% Food & Beverage 32% 5

7 How does the S&OVC work? Member Inputs Model-Based Analytics Member Results Detailed Forecast and Actual Order/Sales Transaction Data Data Review, Cleanup and Validation Accuracy Calculations & Benchmarking Questionnaire Responses on Business Practices and Forecasting Processes Questionnaire Tabulation & Analysis Forecast Accuracy Predictive Model Online Tool Forecast Accuracy Predictions On-Demand 6

8 Members Opinion of Their Performance vs. Their Peers 7

9 Forecast Accuracy Overall Forecast Accuracy Monthly by Item-Location Lag 0 Lag 1 Lag 2 Lag 3 83% 81% 79% 77% 80% 54% 43% 40% 8

10 Forecast Bias Overall Bias Monthly by Item-Location Lag 0 Lag 1 Lag 2 Lag % 19.4% 17.6% 1.5% 1.0% 2.9% 3.3% 4.8% 9

11 % of Units Shipped in Pattern Demand Pattern Influence on FCA and Bias Stable Trending Seasonal/Uplift Intermittent Launch/End Other FCA Bias 18.7% 81% 61% 54% 5.6% 2.9% Members 10

12 Member vs. Consortium Demand Patterns Less Stable More Seasonal/Uplift 11

13 Demand Variability Influence on FCA and Bias FCA BIAS 81% 54% 18.7% 2.9% 0% 10% 20% 30% 40% 50% 60% 70% % of Item Locations with Low Demand Variability 12

14 Demand Velocity Influence on FCA and Bias FCA BIAS 81% 54% 18.7% 2.9% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% % of Item Locations with High Demand Velocity 13

15 % of Units Demanded Product Portfolio Influence Impact of the Long Tail 100% 90% 80% FCA 60% 70% 60% 50% 40% FCA 68% FCA 81% Generally, the higher the concentration of demand, the more item-locations comprise the remaining demand (longer tail), the lower the FCA. 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % of Item Locations 14

16 Forecast Accuracy and Bias Segmentation Framework by Demand Pattern Stable Intermittent Trending Launch/End Seasonal/Uplift Other Lag by Forecast Frequency Monthly Weekly by Supply Chain Level Network Location 15

17 FCA Benchmark Example By Pattern Across Lags Pattern 2 Highest Pattern 1 Member 1 st Group 2 nd Group Lowest 4 th Group 3 rd Group Pattern 3 16

18 FCA Benchmark Example Full Segmentation Variability, Velocity, by Pattern within a Lag Lag 0 Lag 1 Highest 8 8% 10 10% 11 1st Group 25 25% 21 21% 17 2nd Group 16 16% 16 16% 15 3rd Group 5 5% 7 7% 6 4th Group 4 4% 5 5% 11 Lowest 10 10% 9 9% 8 Not Applicable 31 31% 31 31% 31 Lag 0 Lag 1 Lag 2 Lag 3 Total Highest 8 8% 10 10% 11 11% 10 10% 39 10% 1st Group 25 25% 21 21% 17 17% 17 17% 80 20% 2nd Group 16 16% 16 16% 15 15% 15 15% 62 16% 3rd Group 5 5% 7 7% 6 6% 6 6% 24 6% 4th Group 4 4% 5 5% 11 11% 9 9% 29 7% Lowest 10 10% 9 9% 8 8% 11 11% 38 10% Not Applicable 31 31% 31 31% 31 31% 31 31% % 17

19 Example Comparison of Practices Top and Bottom Performer FCA Manual Adjustments (Areas) Reporting Org Frozen Period Items per Analyst Top Down vs. Bottom Up Top Performer Bottom Performer 18

20 Collection of Demand Planning Practices Survey Question Area Background Information Planning Environment Survey Question Area Background Sub- Product Characteristics Customer Characteristics Organization Metrics Planning Practices Execution Process Techniques Inputs Planning Technology 80+ Questions Technology 19

21 Developing Predictive Models Variables Total Data Continuous 38 Survey Binary 162 Continuous 20 Discrete 12 Survey-CALC Continuous 3 Grand Total 235 Variable Type Question Name Binary In what time buckets do you typically forecast? Binary At what level(s) are adjustments to the statistical forecast made? Binary If TimeSeries or Both, what approach(es) do you use? Binary What tool do you primarily use for mainstream forecasting? Discrete How frequently do you typically reset (or optimize) inventory target levels? Binary Do you use customer forecasts in your Division's forecasting process? Continuous Total Items Forecasted for Lag 1 Continuous How many total customers are accounted for in your forecasted item portfolio? Abs Correlation FCA Location 20

22 Example FCA Regression Models Item Portfolio, Network, and Demand Drivers 50%-55% 55%-60% 60%-65% 65%-70% 70%-75% 75%-80% 80%-85% 85%-90% 90%-95% 95%-100% R Squa re % Ite m-loc High Ve locity (First 80% of Dema nd) Avg Loca tions/ SKU % Ite m-loc Low Va ria bility La g 1 La g 2 La g % 95% 90% 85% 80% 75% 70% 45% 65% 40% % Item-Locations HIGH Velocity 35% 30% 25% 20% 15% 5% 0% 0% 10% 20% 30% 40% 50% 60% 70% % Item-Locations LOW Variability 60% 55% 50% 80% 21

23 Example FCA Regression Models Policy Drivers 45%-50% 50%-55% 55%-60% 60%-65% 65%-70% 70%-75% 75%-80% Forecast Accuracy 80% 75% R2 =.75 70% 65% 60% 55% 50% 45% Both Bottom Up Top Down Forecasting Approach Frequency and Type of Manual Adjustments 22

24 Questions? Jeff Metersky VP, SIOP Practice Chainalytics SCoF Booth #305