Intermittent state-space model for demand forecasting

Transcription

1 Intermittent state-space model for demand forecasting 19th IIF Workshop 29th June 2016 Lancaster Centre for Forecasting

2 Motivation Croston (1972) proposes a method for intermittent demand forecasting, mentioning the model: y ι = x t z t He estimates probability using intervals between demands ( 1 q t ). He also assumes that probability is constant between occurrences. Syntetos and Boylan (2001, 2005) show that the conditional expectation of Croston s method is biased. They propose an approximation, that corrects the error.

3 Motivation Snyder (2002) looks at Croston s method in details, claiming that the underlying model is: y ι = x t z t + ɛ t. This model produces both positive and negative data. This is a drawback, so Snyder (2002) proposes a modification, taking exp of non-zero demands.

4 Motivation Shenstone and Hyndman (2005) study several additive models, possibly underlying Croston s method. They argue that any model underlying Croston s method must be: non-stationary, defined on continuous space. They conclude that the implied model has non-realistic properties. They support Snyder (2002) approach with exp.

5 Motivation Teunter et al. (2011) propose a model taking inventory obsolescence into account. The probability of having a demand is decreasing when demand does not occur. Simulation is done, but estimation of parameters is skipped. In the following paper Zied Babai et al. (2014) optimise several methods, including TSB. They use MSE calculated as a difference between the estimate and the actual demand.

6 Motivation Kourentzes (2014) investigates the estimation of Croston, SBA, TSB. He discusses several cost functions. And proposes two new ones, which improves estimation of methods. He finds that optimisation of initial states increases forecasting accuracy.

7 Motivation, overall There is no concise model, underlying all the methods. Because of Shenstone and Hyndman (2005) we believe that it doesn t exist. Intermittent demand methods are disconnected from slow-moving data methods. And we still need to make good decisions about replenishment levels.

8 Universal model

9 Universal model Very general model: y t = o t ỹ t, (1) where o t Bernoulli(p t ) and ỹ t is a statistical model of our choice. This corresponds to Croston s original idea. If o t = 1, for any t, then this is slow-moving data model.

10 Additive state-space model (Snyder, 1985) State-space model: y t = o t (w v t 1 + ɛ t ) v t = F v t 1 + gɛ t, (2) v t 1 vector of states, w is measurement vector, F is transition matrix, g is persistence vector, ɛ t N(0, σ 2 ). Example. iets(a,n,n) with constant probability: y t = o t (l t 1 + ɛ t ) l t = l t 1 + αɛ t, (3) where o t Bernoulli(p).

11 General state-space (based on Hyndman et al. (2008)) State-space model for any ETS: y t = o t (w(v t 1 ) + r(v t 1, ɛ t )). (4) v t = F (v t 1 ) + g(v t 1, ɛ t ) Example. iets(m,ad,n) with constant probability: y t = o t (l t 1 + φb t 1 )(1 + ɛ t ) l t = (l t 1 + φb t 1 )(1 + αɛ t ), (5) b t = φb t 1 (1 + βɛ t ) where o t Bernoulli(p), (1 + ɛ t ) log N(0, σ 2 ).

12 Advantages What are the advantages of such a model? Statistical rationale for intermittent demand; Connection between conventional and intermittent models; Correct estimation of mean; Simpler variance estimation; Prediction intervals;

13 Advantages What else? Both additive and multiplicative ETS models; Any statistical model; Likelihood function; Solution to initialisation and optimisation problems; Model selection.

14 Disadvantages What are the disadvantages of such a model? May need more observations......especially for trend and seasonal models; Derivations in some cases may be messy.

15 iets(m,n,n), constant probability

16 iets(m,n,n), constant probability iets(m,n,n) model has the form: where o t Bernoulli(p). y t = o t l t 1 (1 + ɛ t ) l t = l t 1 (1 + αɛ t ), (6) iets(m,n,n) underlies SES (Hyndman et al., 2008). Conditional expectation: E(y t+h t) = pe(ỹ t+h t) = pw F h 1 v t = pl t.

17 iets(m,n,n), constant probability Conditional variance: h 1 V (y t+h t) = p(1 p)lt 2 + plt 2 σ α 2 (1 + σ 2 ) (1 + α 2 σ 2 ). Messy because of the multiplicative error. j=1

18 iets(m,n,n), constant probability Likelihood can be derived taking probabilities: P (y t o t = 1, θ, σ 2 ) = p 1 1 (1+ɛ y t 2πσ 2 e t )2 2σ 2, P (y t o t = 0, θ, σ 2 ) = 1 p. Product of all the zero and non-zero cases is then: L(θ, σ 2 y t ) = o t=1 p 1 1 (1+ɛ y t 2πσ 2 e t )2 2σ 2 (1 p). (7) o t=0

19 iets(m,n,n), constant probability The concentrated log-likelihood is simple: l(θ, ˆσ 2 y t ) = T ( 1 2 log (2πe) + log (ˆσ 2 )) log(y t ) o t=1 +T 0 log(1 p) + T 1 log p, (8) where T is number of all observations, T 0 is number of zeroes, T 1 number of non-zero demands. The variance of the error estimated using likelihood (8) is: ˆσ 2 = 1 (1 + ɛ t ). T 1 o t=1 The probability can also be derived from (8): p = T 1 T.

20 Example. Intermittent demand ETS(MNN) Series Fitted values Point forecast 95% prediction interval Forecast origin

21 Example. Probabilities Constant probability Series Fitted values Point forecast Forecast origin

22 Simple iets. Sub-conclusion Pretty easy statistical model; Multiplicative ETS is possible and makes more sense than additive; But probability is currently constant;

23 iets(m,n,n), time varying probability, Croston style

24 Croston s iets(m,n,n) ETS(M,N,N) + compound Bernoulli distribution: o t Bernoulli(p t ), where p t = 1 1+q t, q t are intervals between demands. If q t = 0, then p t = 1. Assumption: Probability changes only when demand occurs. State-space model for probabilities: where (1 + ε t ) log N(0, σ 2 q) q t = l q,t 1 (1 + ε t ) l q,t = l q,t 1 (1 + δε t ), (9)

25 Croston s iets(m,n,n) Overall iets(m,n,n) Croston style is: y t = o t l t 1 (1 + ɛ t ) l t = l t 1 (1 + αɛ t ) q t = l q,t 1 (1 + ε t ) l q,t = l q,t 1 (1 + δε t ) (10) (1 + ɛ t ) log N(0, σ 2 ) o t Bernoulli( 1 1+q t ) (1 + ε t ) log N(0, σq). 2 Now it becomes a bit more complicated...

26 Croston s iets(m,n,n) Conditional expectation: ( ) 1 E(y t+h t) = l t E 1 + q t+h t. Not yet simplified: ( ) 1 E(y t+h t) = l t E 1 + l h 1 q,t j=1 (1 + δε t+j)(1 + ε t+h ) t. We feel that this should be close to SBA.

27 Croston s iets(m,n,n) Variance is currently mind blowing... But it should be based on the variance of o t : σ 2 o = p t (1 p t ) Meaning that the conditional variance of y t+h is: ( ) ( )) 1 V (y t+h t) = E 1+q t+h t 1 (1 E 1+q t+h t ( +E ) 1 1+q t+h t l 2 t σ 2 ( 1 + α 2 (1 + σ 2 ) h 1 j=1 (1 + α2 σ 2 ) l 2 t ).

28 Croston s iets(m,n,n) Likelihood however can be done in two stages (assuming demand sizes and intervals are independent): 1. Likelihood for intervals; 2. Likelihood for demands. Both of them are based on lognormal distributions.

29 Croston s iets(m,n,n) Concentrated log-likelihoods. For intervals (first stage): l(θ q, ˆσ 2 q q t ) = T q 2 For demands (second stage): T ( )) q log (2πe) + log 2 (ˆσ q log(q t ), (11) t=1 l(θ, ˆσ 2 y t ) = T ( 1 2 log (2πe) + log (ˆσ 2 )) log(y t ) o t=1 + log(1 p t ) + o t=0 o t=1 log p t, (12)

30 Croston s iets(m,n,n). Example ETS(MNN) Series Fitted values Point forecast 95% prediction interval Forecast origin

31 Croston s iets(m,n,n). Example. Probabilities Croston probability Series Fitted values Point forecast Forecast origin

32 Croston s iets. Sub-conclusion There is a statistical model underlying Croston s method; Conditional expectation should be closer to SBA; Conditional variance can be found analytically; Probabilities are updated only when demand occurs; There are still some problems with derivations.

33 iets(m,n,n), time varying probability, TSB

34 TSB iets(m,n,n) ETS(M,N,N) + compound Bernoulli distribution: o t Bernoulli(p t ), where: p t = l p,t 1 (1 + ξ t ) l p,t = l p,t 1 (1 + δξ t ). (13) p t can be estimated as naïve probability: p t = o t. We want to have conditional Beta(a, b) distribution. But this means that p t (0, 1). We need boundary values!

35 TSB iets(m,n,n) Temporary fix simple transfer function: p t = (1 2κ)p t + κ, where κ is some small number. e.g. κ = This means that p t (κ, 1 κ). So p t Beta(a,b).

36 TSB iets(m,n,n) The fixed TSB iets(m,n,n) is then: y t = o t l t 1 (1 + ɛ t ) l t = l t 1 (1 + αɛ t ) p t = p t κ 1 2κ p t = l p,t 1 (1 + ξ t ) l p,t = l p,t 1 (1 + δξ t ). (14) (1 + ɛ t ) log N(0, σ 2 ) o t Bernoulli(p t ) p t Beta(a,b)

37 TSB iets(m,n,n) Conditional expectation is simpler than in Croston: E(y t+h t) = l t l p,t 1 κ 1 2κ. Conditional variance is based on Bernoulli p t+h t (1 p t+h t ): V (y t+h t) = l p,t 1 κ 1 2κ + l p,t 1 κ 1 2κ l2 t σ 2 ( ) 1 l p,t 1 κ 1 2κ lt 2 h α 2 (1 + σ 2 ) (1 + α 2 σ 2 ). j=1

38 TSB iets(m,n,n) Concentrated log-likelihood in two stages. For the probability (stage 1): T l(θ p, a, b p t ) = (a 1) log(l p,t 1 (1 + ξ t )) +(b 1) t=1 T log(1 l p,t 1 (1 + ξ t )) t=1 T log B(a, b), For the demand sizes (stage 2): l(θ, ˆσ 2 y t ) = T ( 1 2 log (2πe) + log (ˆσ 2 )) log(y t ) o t=1 + log(1 p t ) + o t=0 o t=1 log p t, (15) (16)

39 TSB iets(m,n,n). Example ETS(MNN) Series Fitted values Point forecast 95% prediction interval Forecast origin

40 TSB iets(m,n,n). Example. Probabilities TSB probability Series Fitted values Point forecast Forecast origin

41 TSB iets. Sub-conclusion There is a statistical model underlying TSB; Estimation problem solved; Works fine even with the proposed approximation; p t is unknown, problem with estimation; Problem with distribution of p t ; Multiplicative damped trend could be more appropriate.

42 Real time series example

43 Example on the real data intermittent time series, 2. One product, different branches, daily data, observations each, demand occurrences, 4. Holdout sample of 20 obs, 5. iets using es from smooth package in R ( Stable probability, Croston s probability, TSB probability, 6. Croston s method and TSB, tsintermittent package in R.

44 Example on the real data Method spis sapis ARMSE Complex bias iets, stable % iets, Croston % iets, TSB % Croston s method % TSB method % Zero forecast % Table: Intermittent demand data performance.

45 Conclusions

46 Conclusions We proposed a very simple modification, that can be applied to any model; iets is one of such models; Multiplicative models are available now; Model selection is also available; It can even be done between Stable / Croston / TSB;

47 Conclusions Conditional expectation can be correctly estimated; The same holds for the conditional variance; Prediction intervals for intermittent data; Croston and TSB have underlying iets model; Estimation problem is now solved for them.

48 Finale Thank you for your attention! Ivan Svetunkov, John Boylan Lancaster Centre for Forecasting

49 Croston, J. D., Forecasting and Stock Control for Intermittent Demands. Operational Research Quarterly ( ) 23 (3), 289. Hyndman, R. J., Koehler, A., Ord, K., Snyder, R., Forecasting with Exponential Smoothing. Springer Berlin Heidelberg. Kourentzes, N., On intermittent demand model optimisation and selection. International Journal of Production Economics 156, Shenstone, L., Hyndman, R. J., Stochastic models underlying Croston s method for intermittent demand forecasting. Journal of Forecasting 24 (6), Snyder, R., Forecasting sales of slow and fast moving inventories. European Journal of Operational Research 140 (3), Snyder, R. D., Recursive Estimation of Dynamic Linear

50 Models. Journal of the Royal Statistical Society, Series B (Methodological) 47 (2), Syntetos, A., Boylan, J., On the bias of intermittent demand estimates. International Journal of Production Economics 71 (1-3), Syntetos, A. A., Boylan, J. E., The accuracy of intermittent demand estimates. International Journal of Forecasting 21 (2), Teunter, R. H., Syntetos, A. A., Zied Babai, M., Intermittent demand: Linking forecasting to inventory obsolescence. European Journal of Operational Research 214 (3), Zied Babai, M., Syntetos, A., Teunter, R., Intermittent demand forecasting: An empirical study on accuracy and the risk of obsolescence. International Journal of Production Economics 157 (1),