Comparing Forecast Accuracy of Different Models for Prices of Metal Commodities João Victor Issler (FGV) and Claudia F. Rodrigues (VALE) August, 2012 J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 1 / 19
Stylized Facts on Forecasts and Forecast Combinations Interest in forecasting y t, stationary and ergodic, using information up to h periods prior to t where h is treated as fixed. Risk function is MSE. Then, optimal forecast (Min. MSE) is: E t h (y t ), h step ahead forecast error = y t E t h (y t ). Hendry and Clements (2002): Let fi,t h be the i-th h-step-ahead forecast of y t, i = 1, 2,...,. Then, 1 fi,t h performs very well compared to i =1 individual forecasts f h i,t. This suggests that f h i,t cannot approximate E t h (y t ) very well, since E t h (y t ) is optimal. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 2 / 19
Stylized Facts on Forecasts and Forecast Combinations Forecast combination works from a risk diversification point-of-view (Bates and Granger, 1969, and Timmermann, 2006): if the number of forecasts in the combination is large ( ), the idiosyncratic component of forecast errors is wiped out due to the law of large numbers. However, there is the Forecast Combination Puzzle: consider ω i f h i,t, where ω i < and ω i = 1. In practice, equal weights ω i = 1/ outperform optimal weights designed to outperform it in a MSE sense; Stock and Watson (2006). J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 3 / 19
Issler and Lima (JoE, 2009) Work within a panel-data framework, where, T, with sequential asymptotics: first T with fixed. Then,, written as (T, ) seq. Propose the use of equal weights combination (1/) coupled with an average bias correction term (BCAF): 1 fi,t h B. Propose a new test for the need to do bias correction: H 0 : B = 0. Show that there is no Forecast Combination Puzzle in large samples: for, T large optimal weights have the same limiting MSE of the BCAF. The bad performance of estimated optimal weights is then linked to the curse of dimensionality. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 4 / 19
Issler and Lima (JoE, 2009) They decompose fi,t h, as follows where h is treated as fixed: We can always write: f h i,t = E t h (y t ) + k i + ε i,t, y t = E t h (y t ) + ζ t, with E t h (ζ t ) = 0. Then, fi,t h = y t ζ t + k i + ε i,t, or, fi,t h = y t + k i + η t + ε i,t, where, η t = ζ t, or fi,t h = y t + µ i,t, i = 1, 2,...,, with µ i,t = k i + η t + ε i,t The error µ i,t has a two-way decomposition (Wallace and Hussain (1969), Amemiya (1971), Fuller and Battese (1974)) with a long tradition in the econometrics literature. It depends on h, but this is omitted for notational convenience. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 5 / 19
Issler and Lima (JoE, 2009) Assumptions Time framework: E 1 T 1 T 2 T R P E = T 1 = κ 1 T, R = T 2 T 1 = κ 2 T, P = T T 2 = κ 3 T. Assumption 1 k i, ε i,t and η t are independent of each other for all i and t. Assumption 2 k i is an identically distributed random variable in the cross-sectional dimension, but not necessarily independent, k i i.d.(b, σ 2 k ). (1) J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 6 / 19
Issler and Lima (JoE, 2009) Assumptions Assumption 3 The aggregate shock η t is a stationary and ergodic MA process of order at most h 1, with zero mean and variance σ 2 η <. Assumption 4: Let ε t = (ε 1,t, ε 2,t,... ε,t ) be a 1 vector stacking the errors ε i,t associated with all possible forecasts, where E (ε i,t ) = 0 for all i and t. Then, the vector process {ε t } is assumed to be covariance-stationary and ergodic for the first and second moments, uniformly on. Further, defining as ξ i,t = ε i,t E t 1 (ε i,t ), the innovation of ε i,t, we assume that 1 lim 2 j=1 E ( ξ i,t ξ j,t ) = 0. (2) J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 7 / 19
Issler and Lima (JoE, 2009) Assumption for ested Models Continuous of models (i = 1,.., ) split into M classes (or blocks), each with m nested models: = mm. Let, M = 1 d, and m = d, where 0 d 1. 1 d = 0, all models are non-nested; 2 d = 1, all models are nested and; 3 0 < d < 1 gives rise to 1 d blocks of nested models, all with size d. otice that d is a choice parameter from the point-of-view of the researcher combining forecasts. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 8 / 19
Issler and Lima (JoE, 2009) Assumption for ested Models Partition matrix ( E ( )) ξ i,t ξ j,t into blocks: M main-diagonal blocks, each with m 2 = 2d elements. M 2 M off-diagonal blocks. Index the class by r = 1,.., M, and models within class by s = 1,.., m. Within each block r, we assume that: lim 1 2d d k=1 d s=1 0 lim E ( ξ r,k,t ξ r,s,t ) <, m m 1 m m 2 k=1 s=1 being zero when the smallest nested model is correctly specified. Across any two blocks r and l, r = l, we assume that: m m 1 lim m m 2 k=1 s=1 E ( ξr,k,t ξ l,s,t ) = lim 1 2d d k=1 d s=1 E ( ξr,k,t ξ r,s,t ) = E ( ξr,k,t ξ l,s,t ) = 0. Here, the assumption in the previous page will still hold in the presence of nested models when 0 < d < 1. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 9 / 19
Issler and Lima (JoE, 2009) Main Results If Assumptions 1-4 hold, the following are consistent estimators of k i, B, η t, and ε i,t, respectively: k i = 1 R T 2 t=t 1 +1 f h i,t 1 R T 2 t=t 1 +1 y t, B = 1 k i, plim (T, ) seq η t = 1 fi,t h B y t, ε i,t = f h i,t y t k i η t, ( B B) = 0, ) plim ( k i k i = 0, T plim (T, ) seq ( η t η t ) = 0, plim ( ε i,t ε i,t ) = 0. (T, ) seq J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 10 / 19
Issler and Lima (JoE, 2009) Main Results If Assumptions 1-4 hold, the feasible bias-corrected average forecast 1 fi,t h B obeys: plim (T, ) seq and has a mean-squared error as follows: E [ ( ) 1 fi,t h B = y t + η t = E t h (y t ), plim (T, ) seq Therefore it is an optimal forecasting device. ( ) ] 2 1 fi,t h B y t = σ 2 η. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 11 / 19
Issler and Lima (JoE, 2009) Main Results Consider the sequence of deterministic weights {ω i }, such that ω i = 0, ω i = O ( 1) uniformly, with Then, under Assumptions 1-4: E [ plim (T, ) seq ( ω i f h i,t Therefore it is an optimal forecasting device as well. ω i = 1 and lim ω i k i ) y t ] 2 = σ 2 η. For optimal population weights there is no Forecast Combination Puzzle. ω i = 1. Thus, the Forecast Combination Puzzle must be a consequence of the inability to estimate consistently the optimal population weights. This happens when R is small relative to. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 12 / 19
Issler and Lima (JoE, 2009) Main Results The optimality results above are based on f h i,t = E t h (y t ) + k i + ε i,t, where the bias k i is additive. If the bias is multiplicative as well as additive, i.e., fi,t h = β i E t h (y t ) + k i + ε i,t, ( ) where β i = 1 and β i β, σ 2 β, the BCAF is no longer optimal if β = 1. Optimality can be restored if the BCAF is slightly modified to be ) 1 ( f i,t β k i β where k i and β are consistent estimators of k i and β, respectively., J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 13 / 19
Issler and Lima (JoE, 2009) Main Results Under the null hypothesis H 0 : B = 0, the test statistic: t = B V d (T, ) seq (0, 1), where V is a consistent estimator of the asymptotic variance of B = 1 k i. V is estimated using a cross-section analog of the ewey-west estimator due to Conley (1999), where a natural order in the cross-sectional dimension requires matching spatial dependence to a metric of economic distance. If B = 0, the average forecast 1 fi,t h is an optimal forecasting device. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 14 / 19
Issler and Lima (JoE, 2009) Monte-Carlo The DGP is a stationary AR(1) process: y t = α 0 + α 1 y t 1 + ξ t, with ξ t i.i.d. (0, 1), α 0 = 0, and α 1 = 0.5, One-step-ahead forecasts are generated as: and the bias is generated as: f i,t = α 0 + α 1 y t 1 + k i + ε i,t, k i = βk i 1 + u i, where u i i.i.d.u(a, b), with β = 0.5. The error ε i,t is drawn from a multivariate ormal with zero mean and variance-covariance matrix Σ = (σ ij ), which has zero covariance imposed as long as i j > 2. J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 15 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results R = 50, B = 0.5, σ 2 ξ = σ2 η = 1 Bias MSE BCAF Average Weighted BCAF Average Weighted = 10 mean 0.000 0.391-0.001 1.561 1.697 1.916 = 20 mean 0.000 0.440-0.002 1.286 1.466 2.128 = 40 mean 0.000 0.465 0.000 1.147 1.351 6.094 J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 16 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results R = 50, B = 0, σ 2 ξ = σ2 η = 1 Bias MSE BCAF Average Weighted BCAF Average Weighted = 10 mean 0.000 0.000-0.001 1.561 1.547 1.916 = 20 mean 0.000 0.000-0.002 1.286 1.272 2.128 = 40 mean -0.002 0.000 0.000 1.147 1.133 6.094 J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 17 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results = 10, R = 500, 1, 000, B = 0.5, σ 2 ξ = σ2 η = 1 Bias MSE BCAF Average Weighted BCAF Average Weighted = 10, R = 500 mean 0.000 0.391 0.000 1.532 1.697 1.559 = 10, R = 1, 000 mean 0.000 0.392 0.000 1.535 1.695 1.541 J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 18 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results = 40, R = 2, 000, 4, 000, B = 0.5, σ 2 ξ = σ2 η = 1 Bias MSE BCAF Average Weighted BCAF Average Weighted = 40, R = 2, 000 mean 0.000 0.466 0.000 1.127 1.355 1.149 = 40, R = 4, 000 mean 0.000 0.465 0.000 1.127 1.355 1.137 J.V. Issler and C.F. Rodrigues () Forecast Models for Metal Prices August, 2012 19 / 19