Semi-automatic Non-linear Model Selection

Size: px

Start display at page:

Download "Semi-automatic Non-linear Model Selection"

Collin Hopkins
6 years ago
Views:

1 Semi-automatic Non-linear Model Selection Jennifer L. Castle Institute for New Economic Thinking at the Oxford Martin School, University of Oxford Based on research with David F. Hendry ISF 2013, Seoul Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

2 Background Problems selecting empirical non-linear models Formulating the correct member from infinite class of potential non-linear functions. Non-linear in variables functions also non-linear in parameters. Non-stationarity from stochastic trends and structural breaks. Structural breaks often approximated by non-linearities, & vice versa. Incorrect specification damaging for forecasting, wrongly extrapolating non-existent shifts, or spurious non-linear change. Usual specification and selection issues remain: appropriate set of relevant variables; their correct functional forms and lag lengths; handling location shifts and outliers; endogeneity of contemporaneous variables; measurement accuracy, etc. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

3 Background Two approaches 1 Specific-to-general: including a small subset can lead to model mis-specification, inconsistent parameter estimates, and potential non-constancies. 2 General-to-specific: correlations between candidate variables require that they all be included jointly (Hendry, 2009). Results in more candidate variables than observations. Solution: Automatic model selection software that can handle very large numbers of explanatory variables. E.g. Autometrics, (Doornik, 2009a, and Castle, Doornik, and Hendry, 2011). Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

4 Background Why semi-automatic? Non-linearities found in search are approximations to any realistic non-linear relationship in DGP, and to best parsimonious representation. 1 A dynamically unstable relation might be selected, needs checked by investigator. 2 Encompassing test required against investigator s preferred functions. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

5 Non-linear models Regime shifts changes with sufficient regularities that regimes are re-visited Structural breaks changes in the parameters of the system Approach aims to detect both by modelling regime shifts at the same time as allowing for breaks. Motivates use of SIS/IIS: linear models, where outliers and breaks matter substantively and need to be modelled; non-linear models, where fewer indicators should be found if apparent shifts are captured by non-linearities. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

6 Many possible non-linear functions Class of non-linear in variables functions is vast. Aim to find good approximations to unknown non-linear relation: How closely a single specification represents a wide class of continuous functions. Can the non-linearity be represented in a parsimonious way to obtain relatively precise estimates. Choices include polynomial expansions, trigonometric and hypergeometric series, and squashing functions. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

7 GUM specification Let z i,t denote the set of r linear conditioning variables and w i,t either those variables or their principal components, then the initial general unrestricted specification with s lags is: y t = + + r s r s β i,j z i,t j + κ i,j w i,t j e w i,t j i=1 r j=0 s θ i,j wi,t j 2 + i=1 r j=0 s γ i,j wi,t j 3 + i=1 j=0 i=1 j=0 j=1 T δ i 1 {i=t} + ɛ t i=1 s λ j y t j Such a formulation leads to N = 4r(s + 1) + s + T candidate regressors, so the approach is bound to generate N > T. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

8 Model selection in action Empirical example: UK real wages over the past century and a half. Objective: show that all aspects must be modelled jointly for a coherent economic model, including all substantively relevant variables, any dynamics, outliers and breaks, and non-linearities. 1 Explore non-linearities by undertaking model selection for several non-linear functions, retaining linearity in parameters. 2 Semi-automated approach: attempt to encompass selected model with a non-linear wage-price spiral term. 3 Encompassing 4 Exogeneity 5 Extended data set to forecast the last 7 years of real wages. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

9 Wages and prices w p w p w p 0.15 (w p) Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

10 Productivity, ULCs and unemployment 3.0 y l 0.3 ulc p Ur Ur Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

11 Non-linearity test The Castle and Hendry (2010) non-linearity index test applied to a linear model of real wage growth, where the regressors include an intercept, (w p) t i and (ulc p) t i for i = 1, 2 and (y l) t j, U r,t j and p t j for j = 0, 1, 2. The test is significant at p = with F(36, 91) = Castle and Hendry (2009) also found the index test to be significant. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

12 Wage-price spirals Castle and Hendry (2009) found evidence for a non-linear real wage reaction to inflation: f t = ( p t ) fna p Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

13 The extended and updated CH model (w p) t = (0.002) (0.126) (ft pt ) (0.045) (y l) t (0.048) (y l) t (ulc p) t Ur,t I (0.010) (0.044) (0.013) (0.013) I (0.006) (I I1943 I1944 I1945 ) (0.009) (I I1977 ) (1) R 2 = 0.733; σ = 1.24%; SIC = 5.66; χ 2 nd (2) = 2.21; F ar (2, 130) = 0.766; F arch (1, 140) = 0.109; F het (13, 126) = 0.794; F reset (2, 130) = 0.106; F chow (7, 132) = 1.354; T = Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

14 The extended and updated CH model (w p) t = (0.002) (0.126) (ft pt ) (0.045) (y l) t (0.048) (y l) t (ulc p) t Ur,t I (0.010) (0.044) (0.013) (0.013) I (0.006) (I I1943 I1944 I1945 ) (0.009) (I I1977 ) (1) R 2 = 0.733; σ = 1.24%; SIC = 5.66; χ 2 nd (2) = 2.21; F ar (2, 130) = 0.766; F arch (1, 140) = 0.109; F het (13, 126) = 0.794; F reset (2, 130) = 0.106; F chow (7, 132) = 1.354; T = Update is close to original despite data revisions, and is relatively constant over the Great Recession. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

15 Model fit, residuals and forecast errors 0.15 (w p) Fitted a 2 scaled residuals forecast errors b step forecasts (w p) Residual density N(0,1) c 0.4 d Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

16 An approximating model f t is a variant of an LSTAR in π 2 t where π t = 100 p t (annual inflation measured as a percentage), given by (scaling to the same mean and range as f t ): Lp t = 2 ( 1 + exp( γπ 2 t ) ) 1 2 so the approximation in the Taylor expansion becomes: α 1 p t + α 2 ( p t ) 3 + α 3 ( p t ) 4 Also included the most significant non-linear function of the other regressors, U 2 r,t. Selection at 1%. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

17 An approximating model Wage-price spiral as an LSTAR approximation (w p) t = (0.003) (0.050) (y l) t (0.055) (y l) t (0.013) (ulc p) t Ur,t (0.042) (0.80) U2 r,t Ur,t (0.050) (1.63) ( pt ) (5.44) ( pt ) pt (0.045) (0.03) 2 p t I I (0.013) (0.013) (0.013) I1977 (0.013) I1945 R 2 = 0.747; σ = 1.23%; SIC = 5.55; (2) χ 2 nd (2) = 0.88; F ar (2, 124) = 0.63; F arch (1, 139) = 0.18; F het (19, 117) = 1.12; F chow (7, 126) = 1.61; T = Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

18 An approximating model Both non-linear terms in inflation are highly significant, and the fit and mis-specification tests are similar to (1). I 1922, I 1939 and I 1942 eliminated by U 2 r,t, ( p t ) 3 and ( p t ) 4. U r,t is intrinsically positive, so combined term, 0.18U r,t (1 13.7U r,t ), is negative till the unemployment rate exceeds 7.25% then becomes positive. Such an effect could represent movements along the marginal product curve, raising real wages of those still employed as employment fell, from more capital per worker and the unemployment of less productive workers. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

19 A nesting model (w p) t = (0.003) (0.047) (y l) t (0.053) (y l) t (0.011) (ulc p) t Ur,t (0.039) (0.79) U2 r,t Ur,t (ft pt) (0.050) (0.14) I I I (0.013) (0.013) (0.013) (0.012) R 2 = 0.747; σ = 1.22%; SIC = 5.61; T = ; χ 2 nd (2) = 0.54; F ar (2, 126) = 0.96; F arch (1, 139) = 0.06; F het (15, 121) = 1.26; F reset (2, 126) = 0.28; F chow (7, 128) = (0.031) 2 p t I1977 (3) The wage-price spiral term is not sufficient to model all non-linearity, but does explain that for impact of inflation on real-wage growth. Some restricted dummies drop out. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

20 Model fit, residuals and forecast errors 0.15 (w p) ^ (w p) a scaled residuals forecast errors 2 b Residual density N(0,1) c Residual correlogram d Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

21 An LSTAR model Replacing f t p t by Lp p t, non-linear estimation leads to γ = Lp generates almost identical behaviour to f t (correlation 0.96). (w p) t = (0.003) (0.049) (y l) t (0.053) (y l) t (0.013) (ulc p) t Ur,t (0.042) (0.23) pt (0.80) U2 r,t ( 1 + exp (0.049) 2Ur,t ( (0.023) π2 )) I I I (0.013) (0.013) (0.013) (0.013) (0.24) 2 p t (0.24) pt 1 R 2 = 0.752; σ = 1.22%; SIC = 5.60; T = ; χ 2 nd (2) = 0.31; F ar (2, 124) = 1.26; F arch (1, 139) = 0.14; F het (15, 121) = 1.81 ; F chow (7, 126) = I1977 (4) Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

22 f and Lp f 0.00 a Lp 0.00 b f 0.00 t p Lp 0.00 t c d p Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

23 Encompassing Neither model (4) and (2) encompasses the other: Test Model 1 vs. Model 2 Model 2 vs. Model 1 Cox N(0,1) Joint Model F(2,125) = 4.08 F(1,125) = 10.9 Neither Lp t p is significant if added to (3), nor f t p t when added to (4), so they are close substitutes. Despite p t entering ( w t p t ), real wages are primarily determined by forces different from nominal prices, consistent with the Classical dichotomy : in particular, the impact of p t on real wages is zero at high inflation. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

24 Alternative modelling of regime change Re-estimated model proposed by Nielsen (2009) (w p) t = (0.002) (0.126) (f p) t (0.045) (y l) t (ulc p) t (0.013) (0.045) 2Ur,t ( ) I { } U 1 r (0.0001) t ( ) I { } U 1 r ( ) t 2 ( I{ } log (U r ) ) t (0.008) (0.0009) ( I{ } log (U r ) ) t (0.012) I1918t I1940t I WWIIt (0.012) (0.006) (0.008) I7577t R 2 = 0.783; σ = 1.13%; SC = 5.77; T = ; χ 2 nd (2) = 0.53; F ar (2, 126) = 0.089; F arch (1, 139) = 0.003; F het (19, 119) = 1.093; F reset (2, 126) = 2.228; F chow (7, 128) = (5) Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

25 Alternative modelling of regime change Log-likelihood for equation augmenting (5) by additional regressors from (3) and (1) is 446.4, and for (3) is 428.0, so F(4, 123) = The regime-shift variables matter, and remain relevant over the great recession, as curtailing their influence to 2004 leads to marked deterioration in RMSFE. Neither model encompasses the other, as a test of those additional regressors yields F(5, 123) = Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

26 IIS and SIS Extension of IIS to step-indicator saturation (SIS): adding a complete set of step indicators S 1 = { 1 {t j}, j = 1,..., T }, where 1 {t j} = 1 for observations up to j, and zero otherwise. Step indicators are the cumulation of impulse indicators up to each next observation. IIS: Impulses SIS: Step Shifts SIS has the correct null retention frequency in constant conditional models for a nominal test size of α. The approximate alternative retention-frequency function has been derived analytically for simple models, and shows higher probabilities of retaining location shifts. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

27 Illustrating SIS when no location shifts Split-sample search by SIS at 1%. Block Indicators included initially Indicators retained Selected model: actual and fitted 12.5 actual fitted Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

28 Illustrating SIS when no location shifts Split-sample search by SIS at 1%. Indicators included initially Indicators retained Selected model: actual and fitted actual fitted 12.5 Block Block Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

29 Illustrating SIS when no location shifts 1.0 Indicators included initially 1.0 Indicators retained Selected model: actual and fitted actual fitted 12.5 Block Block Final T = 100, and no shifts, retains 2 significant steps, so lose 2 degrees of freedom but could be combined to one dummy. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

30 SIS for a single location shift Split-sample search in SIS when a shift occurs. Block 1 Indicators included initially Indicators retained Selected model: actual and fitted 15 actual fitted 10 5 Block 2 Final Initially retains last step as mean shifts down, then finds location shift, so eliminates the now redundant indicator: just one step needed. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

31 Encompassing model Checking robustness by applying SIS (w p) t = (0.003) (0.042) (y l) t (0.034) 2 (y l) t (0.028) (w p y + l µ) t Ur,t (0.034) (0.68) U2 r,t Ur,t (0.045) (0.012) (ft pt ) S S S (0.011) (0.015) (0.011) (0.008) 0.036I (0.011) (0.006) (I I1943 I1944 I1945 ) R 2 = 0.820; σ = 1.04%; SIC = 5.85; (0.029) 2 p t 1 (S2011 S1946) ur,t I1977 (6) (0.011) χ 2 nd (2) = 2.26; F ar (2, 123) = 0.39; F arch (1, 139) = 0.49; F reset (2, 124) = 2.28; F het (20, 116) = 0.82; F chow (7, 125) = 0.95; T = (w p y + l) replaces (ulc p) adjusted for changes in hours ( µ is sample mean of (w p y + l)). u r,t = log(u r,t ) (e.g.) S 1939 is step indicator: unity till 1939 and zero thereafter. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

32 Encompassing model 0.15 (w p) a ^ (w p) 2 scaled residuals forecast error b step forecasts (w p) Residual density N(0,1) c 0.4 d Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

33 Testing exogeneity (6) encompasses the previous models All mis-specification tests are insignificant Most variables in common with (3) have similar coefficients, other than a stronger and more rapid feedback of almost 0.18 from the previous labour share, and replacing (y l) t 2 by 2 (y l) t 1, as well as switching from pure impulse dummies to a mixture of steps and impulses. Two of the variables from (5) are also retained, so an interaction of a step shift with a variable matters as well. The main role of the step indicators is explaining the much higher average growth rate of real wages post war (1.8% p.a., versus 0.7% p.a. pre-1945), even though (y l) is included and displays a similar pattern. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

34 Castle Table: (Oxford) IIS super-exogeneity Semi-automatic Non-linear tests Model of (3) Selection and SIS tests of (6). ISF / 37 Testing exogeneity IIS/SIS is used to test exogeneity of the conditioning variables, Hendry and Santos (2010). Under null of super exogeneity, parameters in the conditional model are invariant to shifts in the marginal models, so indicators in the latter should not enter the former. A VAR in w p, y l, p and U r was selected with IIS/SIS, and additional impulse indicators in the 3 marginal models were then tested for significance in (3)/(6). Super exogeneity tests Variable null distribution IIS test statistic null distribution SIS test statistic (y l) t F(11,117) 1.16 F(2,123) 0.77 p t F(11,117) 1.22 F(7,118) 1.87 U r,t F(9,118) 1.05 F(14,111) 1.37 Joint F(16,112) 1.22 F(20,105) 1.41

35 Forecasts for real wages 4.85 Equation (3), No IC 4.85 Equation (3) with IC Equation (5), no IC Equation (5) with IC Equation (6), no IC Equation (6) with IC Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

36 Forecasting RMSFEs of forecasts of (w p) with and without intercept corrections Equation σ No IC IC (3) 1.22% 1.31% 1.25% (5) 1.13% 1.23% 1.04% (6) 1.04% 1.05% 1.00% RW 2.23% 1.57% 1.54% VAR 1.67% 2.37% 1.54% Table: RMSFEs of forecasts of (w p) T+h with and without intercept corrections, with in-sample equation standard error for comparison. IC is average residual over Most complicated model has smallest RMSFE. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

37 Conclusions Joint modelling of dynamics, location shifts, relevant variables and non-linearities essential. Automatic model selection despite N > T seems a viable approach to tackling all complications jointly. Wage-price spiral adds a unit root to the wage-price process. Can be approximated in several ways. Real wages are primarily determined by forces different from nominal prices, consistent with the Classical dichotomy. A general polynomial led to an additional non-linearity in unemployment; real wages rose with unemployment beyond about 7.25%, probably rising marginal productivity rather than wage bargaining. Consistent with involuntary unemployment, as no evidence of any reverse relation of high real wages causing unemployment was found in Hendry (2001). Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

38 Conclusions Empirical evidence for non-linear adjustments of real wages to inflation. Reaction based on exogenous variable. IIS did not preclude finding non-linearites, and non-linearities removed indicators found in linear specifications. Not removing the large outliers could hide the presence of other variables, including the non-linearities. Most complicated model produced the smallest 1-step forecast errors rebuts parsimony. All forecasts benefitted from intercept corrections setting their forecasts back on track at the forecast origin. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

39 References I Castle, J. L., J. A. Doornik, and D. F. Hendry (2011). Evaluating automatic model selection. Journal of Time Series Econometrics 3 (1), DOI: / Castle, J. L. and D. F. Hendry (2009). The long-run determinants of UK wages, Journal of Macroeconomics 31, Castle, J. L. and D. F. Hendry (2010). A low-dimension portmanteau test for non-linearity. Journal of Econometrics 158(2), Castle, J. L. and D. F. Hendry (2011). Automatic selection of non-linear models. In L. Wang, H. Garnier, and T. Jackman (Eds.), System Identification, Environmental Modelling and Control, pp New York: Springer. Doornik, J. A. (2009a). Autometrics. In J. L. Castle and N. Shephard (Eds.), The Methodology and Practice of Econometrics: A Festschrift in Honour of David F. Hendry, pp Oxford: Oxford University Press. Doornik, J. A. (2009b). Econometric model selection with more variables than observations. Working paper, Economics Department, University of Oxford. Granger, C. W. J. and T. Teräsvirta (1993). Modelling Nonlinear Economic Relationships. Oxford: Oxford University Press. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

40 References II Hendry, D. F. (2001). Modelling UK inflation, Journal of Applied Econometrics 16, Hendry, D. F. (2009). The methodology of empirical econometric modeling: Applied econometrics through the looking-glass. In T. C. Mills and K. D. Patterson (Eds.), Palgrave Handbook of Econometrics, pp Basingstoke: Palgrave MacMillan. Hendry, D. F. and S. Johansen (2013). Model discovery and Trygve Haavelmo s legacy. Econometric Theory, forthcoming. Hendry, D. F. and H.-M. Krolzig (2005). The properties of automatic Gets modelling. Economic Journal 115, C32 C61. Hendry, D. F. and C. Santos (2010). An automatic test of super exogeneity. In M. W. Watson, T. Bollerslev, and J. Russell (Eds.), Volatility and Time Series Econometrics, pp Oxford: Oxford University Press. Nielsen, H. B. (2009). Comment on the long-run determinants of UK wages, Journal of Macroeconomics 31, White, H. (1980). A heteroskedastic-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48, Castle (Oxford) Semi-automatic Non-linear Model Selection ISF / 37

IIF Economic Forecasting Summer School 2018, Boulder Colorado, USA

IIF Economic Forecasting Summer School 2018, Boulder Colorado, USA Dr. Jennifer L. Castle, David F. Hendry, and Dr. J. James Reade Economics Department, Oxford and Reading Universities, UK. September 30,