Exponential smoothing and non-negative data

Transcription

1 Exponential smoothing and non-negative data 1 Exponential smoothing and non-negative data Muhammad Akram Rob J Hyndman J Keith Ord Business & Economic Forecasting Unit

2 Exponential smoothing and non-negative data 2 Outline 1 Exponential smoothing models 2 Problems with some of the models 3 A new model for positive data 4 Conclusions

3 Exponential smoothing and non-negative data Exponential smoothing models 3 Outline 1 Exponential smoothing models 2 Problems with some of the models 3 A new model for positive data 4 Conclusions

4 Exponential smoothing and non-negative data Exponential smoothing models 4 Problem Most forecasting methods in business are based on exponential smoothing.

5 Exponential smoothing and non-negative data Exponential smoothing models 4 Problem Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative.

6 Exponential smoothing and non-negative data Exponential smoothing models 4 Problem Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don t always work!

7 Exponential smoothing and non-negative data Exponential smoothing models 4 Problem Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don t always work! 1 They can produce negative forecasts Forecasts from ETS(A,A,N) Time

8 Exponential smoothing and non-negative data Exponential smoothing models 4 Problem Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don t always work! 1 They can produce negative forecasts 2 They can produce infinite forecast variance Forecasts from ETS(A,M,N) Time

9 Exponential smoothing and non-negative data Exponential smoothing models 4 Problem Most forecasting methods in business are based on exponential smoothing. Most time series in business are inherently non-negative. But exponential smoothing models for non-negative data don t always work! 1 They can produce negative forecasts 2 They can produce infinite forecast variance 3 They can converge almost surely to zero. y Simulation from ETS(M,N,N) Time

10 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m

11 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal)

12 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing

13 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,N,N): Simple exponential smoothing with additive errors

14 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,A,N): Holt s linear method with additive errors

15 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,A,A): Additive Holt-Winters method with additive errors

16 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(M,A,M): Multiplicative Holt-Winters method with multiplicative errors

17 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing ETS(A,A d,n): Damped trend method with additive errors

18 Exponential smoothing and non-negative data Exponential smoothing models 5 Taxonomy of models Seasonal Component Trend N A M Component (None) (Additive) (Multiplicative) N (None) N,N N,A N,M A (Additive) A,N A,A A,M A d (Additive damped) A d,n A d,a A d,m M (Multiplicative) M,N M,A M,M M d (Multiplicative damped) M d,n M d,a M d,m General notation ETS(Error,Trend,Seasonal) ExponenTial Smoothing There are 30 separate models in the ETS framework

19 Exponential smoothing and non-negative data Exponential smoothing models 6 Innovations state space model No trend or seasonality and multiplicative errors Example: ETS(M,N,N) y t = l t 1 (1 + ε t ) l t = αy t + (1 α)l t 1 0 α 1 ε t is white noise with mean zero.

20 Exponential smoothing and non-negative data Exponential smoothing models 6 Innovations state space model No trend or seasonality and multiplicative errors Example: ETS(M,N,N) y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ) 0 α 1 ε t is white noise with mean zero.

21 Exponential smoothing and non-negative data Exponential smoothing models 6 Innovations state space model No trend or seasonality and multiplicative errors Example: ETS(M,N,N) y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ) 0 α 1 ε t is white noise with mean zero. All exponential smoothing models can be written using analogous state space equations.

22 Exponential smoothing and non-negative data Exponential smoothing models 7 New book! 1 3 Springer Series in Statistics Rob J. Hyndman Anne B. Koehler J. Keith Ord Ralph D. Snyder Forecasting with Exponential Smoothing The State Space Approach State space modeling framework Prediction intervals Model selection Maximum likelihood estimation All the important research results in one place with consistent notation Many new results 375 pages but only US$39.95.

23 Exponential smoothing and non-negative data Exponential smoothing models 7 New book! 1 3 Springer Series in Statistics Rob J. Hyndman Anne B. Koehler J. Keith Ord Ralph D. Snyder Forecasting with Exponential Smoothing The State Space Approach State space modeling framework Prediction intervals Model selection Maximum likelihood estimation All the important research results in one place with consistent notation Many new results 375 pages but only US$

24 Exponential smoothing and non-negative data Problems with some of the models 8 Outline 1 Exponential smoothing models 2 Problems with some of the models 3 A new model for positive data 4 Conclusions

25 Exponential smoothing and non-negative data Problems with some of the models 9 Negative forecasts Forecasts from ETS(A,A,N) Time Could solve by taking logs or some other Box-Cox transformation. However, this limits models to be additive in the transformed space.

26 Exponential smoothing and non-negative data Problems with some of the models 9 Negative forecasts Forecasts from ETS(A,A,N) Time Could solve by taking logs or some other Box-Cox transformation. However, this limits models to be additive in the transformed space. Could solve by only using multiplicative models. But these can have other problems.

27 Exponential smoothing and non-negative data Problems with some of the models 10 Infinite forecast variances ETS(A,M,N) model y t = l t 1 b t 1 + ε t l t = l t 1 b t 1 + αε t b t = b t 1 + βε t /l t 1. l 0 = 0.1 b 0 = 1 α = 0.1 β = 0.05 σ = Forecasts from ETS(A,M,N)

28 Exponential smoothing and non-negative data Problems with some of the models 10 Infinite forecast variances y b ETS(A,M,N) model y t = l t 1 b t 1 + ε t l t = l t 1 b t 1 + αε t b t = b t 1 + βε t /l t 1. l 0 = 0.1 b 0 = 1 α = 0.1 β = 0.05 σ = 1 ell Time Time Time

29 Exponential smoothing and non-negative data Problems with some of the models 11 Infinite forecast variances Suppose ε t has positive density at 0 For ETS models (A,M,N), (A,M,A), (A,M d,n), (A,M d,a), (A,M,M), (A,M d,m), (M,M,A) and (M,M d,a): V(y n+h x n ) = for h 3; E(y n+h x n ) is undefined for h 3.

30 Exponential smoothing and non-negative data Problems with some of the models 11 Infinite forecast variances Suppose ε t has positive density at 0 For ETS models (A,M,N), (A,M,A), (A,M d,n), (A,M d,a), (A,M,M), (A,M d,m), (M,M,A) and (M,M d,a): V(y n+h x n ) = for h 3; E(y n+h x n ) is undefined for h 3. For ETS models (A,N,M), (A,A,M) and (A,A d,m): V(y n+h x n ) = for h m + 2; E(y n+h x n ) is undefined for h m + 2.

31 Exponential smoothing and non-negative data Problems with some of the models 11 Infinite forecast variances Suppose ε t has positive density at 0 For ETS models (A,M,N), (A,M,A), (A,M d,n), (A,M d,a), (A,M,M), (A,M d,m), (M,M,A) and (M,M d,a): V(y n+h x n ) = for h 3; E(y n+h x n ) is undefined for h 3. For ETS models (A,N,M), (A,A,M) and (A,A d,m): V(y n+h x n ) = for h m + 2; E(y n+h x n ) is undefined for h m + 2. These problems occur regardless of the sample space of {y t }.

32 Exponential smoothing and non-negative data Problems with some of the models 12 Convergence to zero y y y ETS(M,N,N) model y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ) Time l 0 = 10 α = 0.3 σ = 0.3 with truncated Gaussian errors Time Time

33 Exponential smoothing and non-negative data Problems with some of the models 13 Convergence to zero ETS(M,N,N) model y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ), 0 < α 1

34 Exponential smoothing and non-negative data Problems with some of the models 13 Convergence to zero ETS(M,N,N) model y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ), 0 < α 1 δ t = 1 + ε t has mean 1 and variance σ 2.

35 Exponential smoothing and non-negative data Problems with some of the models 13 Convergence to zero ETS(M,N,N) model y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ), 0 < α 1 δ t = 1 + ε t has mean 1 and variance σ 2. δ t are iid with positive distribution such as truncated normal, lognormal, gamma, etc.

36 Exponential smoothing and non-negative data Problems with some of the models 13 Convergence to zero ETS(M,N,N) model y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ), 0 < α 1 δ t = 1 + ε t has mean 1 and variance σ 2. δ t are iid with positive distribution such as truncated normal, lognormal, gamma, etc.

37 Exponential smoothing and non-negative data Problems with some of the models 13 Convergence to zero ETS(M,N,N) model y t = l t 1 (1 + ε t ) l t = l t 1 (1 + αε t ), 0 < α 1 δ t = 1 + ε t has mean 1 and variance σ 2. δ t are iid with positive distribution such as truncated normal, lognormal, gamma, etc. l t = l 0 (1 + αε 1 )(1 + αε 2 ) (1 + αε t ) = l 0 U t, where U t = U t 1 (1 + αε t ) and U 0 = 1. Thus U t is a non-negative product martingale.

38 Exponential smoothing and non-negative data Problems with some of the models 14 Convergence to zero Kakutani s Theorem says that l t will converge to 0 almost surely if ε t has mean zero and is non-degenerate.

39 Exponential smoothing and non-negative data Problems with some of the models 14 Convergence to zero Kakutani s Theorem says that l t will converge to 0 almost surely if ε t has mean zero and is non-degenerate. Consequently, all sample paths for y t converge to 0 almost surely.

40 Exponential smoothing and non-negative data Problems with some of the models 14 Convergence to zero Kakutani s Theorem says that l t will converge to 0 almost surely if ε t has mean zero and is non-degenerate. Consequently, all sample paths for y t converge to 0 almost surely. Similar results follow for all purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,M d,n) and (M,M d,m).

41 Exponential smoothing and non-negative data Problems with some of the models 15 Four model classes Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,M d,n) and (M,M d,m). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,A d,n) and (A,A d,a). Class X: Mixed models: (A,M, ), (A,M d, ), (A,,M), (M,M,A), (M,M d,a). (11 models) Class Y: Mixed models: (M,A, ), (M,A d, ) or (M,N,A). (7 models)

42 Exponential smoothing and non-negative data Problems with some of the models 15 Four model classes Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,M d,n) and (M,M d,m). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,A d,n) and (A,A d,a). Class X: Mixed models: (A,M, ), (A,M d, ), (A,,M), (M,M,A), (M,M d,a). (11 models) Class Y: Mixed models: (M,A, ), (M,A d, ) or (M,N,A). (7 models) Only Class M can guarantee a sample space restricted to the positive half-line.

43 Exponential smoothing and non-negative data Problems with some of the models 15 Four model classes Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,M d,n) and (M,M d,m). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,A d,n) and (A,A d,a). Class X: Mixed models: (A,M, ), (A,M d, ), (A,,M), (M,M,A), (M,M d,a). (11 models) Class Y: Mixed models: (M,A, ), (M,A d, ) or (M,N,A). (7 models) Only Class M can guarantee a sample space restricted to the positive half-line. All Class M models converge to 0 if E(ε) = 0

44 Exponential smoothing and non-negative data Problems with some of the models 15 Four model classes Class M: Purely multiplicative models: (M,N,N), (M,N,M), (M,M,N), (M,M,M), (M,M d,n) and (M,M d,m). Class A: Purely additive models: (A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,A d,n) and (A,A d,a). Class X: Mixed models: (A,M, ), (A,M d, ), (A,,M), (M,M,A), (M,M d,a). (11 models) Class Y: Mixed models: (M,A, ), (M,A d, ) or (M,N,A). (7 models) Only Class M can guarantee a sample space restricted to the positive half-line. All Class M models converge to 0 if E(ε) = 0 All Class X models have infinite forecast variance for h m + 2 where m is the seasonal period.

45 Exponential smoothing and non-negative data A new model for positive data 16 Outline 1 Exponential smoothing models 2 Problems with some of the models 3 A new model for positive data 4 Conclusions

46 Exponential smoothing and non-negative data A new model for positive data 17 New models Let δ t = (1 + ε t ) be a positive random variable. METS(M,N,N) model y t = l t 1 δ t l t = l t 1 δ α t

47 Exponential smoothing and non-negative data A new model for positive data 17 New models Let δ t = (1 + ε t ) be a positive random variable. METS(M,N,N) model y t = l t 1 δ t l t = l t 1 δ α t log(y t ) = log(l t 1 ) + log(δ t ) log(l t ) = log(l t 1 ) + α log(δ t )

48 Exponential smoothing and non-negative data A new model for positive data 17 New models Let δ t = (1 + ε t ) be a positive random variable. METS(M,N,N) model y t = l t 1 δ t l t = l t 1 δ α t log(y t ) = log(l t 1 ) + log(δ t ) log(l t ) = log(l t 1 ) + α log(δ t ) Thus the log-transformed model is identical to Gaussian ETS(A,N,N) model if δ t is lognormal with median 1.

49 Exponential smoothing and non-negative data A new model for positive data 18 Long term forecast behaviour METS(M,N,N; LN) model y t = l t 1 δ t l t = l t 1 δ α t, δ t logn(µ, ω)

50 Exponential smoothing and non-negative data A new model for positive data 18 Long term forecast behaviour METS(M,N,N; LN) model y t = l t 1 δ t l t = l t 1 δ α t, δ t logn(µ, ω) Range E(δ α t ) E(δ α/2 t ) E(y h ) V(y h ) µ + αω < 0 < 1 < 1 Decreasing Decreasing µ + αω = 0 < 1 < 1 Decreasing Finite αω < µ < αω/2 < 1 < 1 Decreasing Increasing µ + αω/2 = 0 = 1 < 1 Finite Increasing αω/2 < µ < αω/4 > 1 < 1 Increasing Increasing µ + αω/4 = 0 > 1 = 1 Increasing Increasing µ + αω/4 > 0 > 1 > 1 Increasing Increasing

51 (a) (c) Exponential smoothing and non-negative data A new model for positive data 19 Long term forecast behaviour t t (b) (d) t t METS(M,N,N;LN) δ t logn(µ, ω): (a) µ = αω/4 (b) µ = 0 (c) µ = αω/4 (d) µ = 3αω/8 (e) µ = αω/2 (f) µ = 3αω/4 (e) (f) t t

52 Exponential smoothing and non-negative data Conclusions 20 Outline 1 Exponential smoothing models 2 Problems with some of the models 3 A new model for positive data 4 Conclusions

53 Exponential smoothing and non-negative data Conclusions 21 Conclusions 1 The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line.

54 Exponential smoothing and non-negative data Conclusions 21 Conclusions 1 The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line. 2 This is not a concern for most short-term forecasting.

55 Exponential smoothing and non-negative data Conclusions 21 Conclusions 1 The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line. 2 This is not a concern for most short-term forecasting. 3 An alternative formulation has been proposed that avoids these problems.

56 Exponential smoothing and non-negative data Conclusions 21 Conclusions 1 The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line. 2 This is not a concern for most short-term forecasting. 3 An alternative formulation has been proposed that avoids these problems. 4 The ergodic behaviour of the alternative formulation requires careful parameter choice.

57 Exponential smoothing and non-negative data Conclusions 21 Conclusions 1 The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line. 2 This is not a concern for most short-term forecasting. 3 An alternative formulation has been proposed that avoids these problems. 4 The ergodic behaviour of the alternative formulation requires careful parameter choice.

58 Exponential smoothing and non-negative data Conclusions 21 Conclusions 1 The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line. 2 This is not a concern for most short-term forecasting. 3 An alternative formulation has been proposed that avoids these problems. 4 The ergodic behaviour of the alternative formulation requires careful parameter choice. Paper:

59 Exponential smoothing and non-negative data Conclusions 21 Conclusions 1 The standard exponential smoothing state space models all have theoretical problems when constrained to the positive half line. 2 This is not a concern for most short-term forecasting. 3 An alternative formulation has been proposed that avoids these problems. 4 The ergodic behaviour of the alternative formulation requires careful parameter choice. Paper: Book: