Semi-automatic Non-linear Model Selection

Semi-automatic Non-linear Model Selection Jennifer L. Castle Institute for New Economic Thinking at the Oxford Martin School, University of Oxford Based on research with David F. Hendry ISF 2013, Seoul Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 1 / 37

Background Problems selecting empirical non-linear models Formulating the correct member from infinite class of potential non-linear functions. Non-linear in variables functions also non-linear in parameters. Non-stationarity from stochastic trends and structural breaks. Structural breaks often approximated by non-linearities, & vice versa. Incorrect specification damaging for forecasting, wrongly extrapolating non-existent shifts, or spurious non-linear change. Usual specification and selection issues remain: appropriate set of relevant variables; their correct functional forms and lag lengths; handling location shifts and outliers; endogeneity of contemporaneous variables; measurement accuracy, etc. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 2 / 37

Background Two approaches 1 Specific-to-general: including a small subset can lead to model mis-specification, inconsistent parameter estimates, and potential non-constancies. 2 General-to-specific: correlations between candidate variables require that they all be included jointly (Hendry, 2009). Results in more candidate variables than observations. Solution: Automatic model selection software that can handle very large numbers of explanatory variables. E.g. Autometrics, (Doornik, 2009a, and Castle, Doornik, and Hendry, 2011). Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 3 / 37

Background Why semi-automatic? Non-linearities found in search are approximations to any realistic non-linear relationship in DGP, and to best parsimonious representation. 1 A dynamically unstable relation might be selected, needs checked by investigator. 2 Encompassing test required against investigator s preferred functions. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 4 / 37

Non-linear models Regime shifts changes with sufficient regularities that regimes are re-visited Structural breaks changes in the parameters of the system Approach aims to detect both by modelling regime shifts at the same time as allowing for breaks. Motivates use of SIS/IIS: linear models, where outliers and breaks matter substantively and need to be modelled; non-linear models, where fewer indicators should be found if apparent shifts are captured by non-linearities. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 5 / 37

Many possible non-linear functions Class of non-linear in variables functions is vast. Aim to find good approximations to unknown non-linear relation: How closely a single specification represents a wide class of continuous functions. Can the non-linearity be represented in a parsimonious way to obtain relatively precise estimates. Choices include polynomial expansions, trigonometric and hypergeometric series, and squashing functions. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 6 / 37

GUM specification Let z i,t denote the set of r linear conditioning variables and w i,t either those variables or their principal components, then the initial general unrestricted specification with s lags is: y t = + + r s r s β i,j z i,t j + κ i,j w i,t j e w i,t j i=1 r j=0 s θ i,j wi,t j 2 + i=1 r j=0 s γ i,j wi,t j 3 + i=1 j=0 i=1 j=0 j=1 T δ i 1 {i=t} + ɛ t i=1 s λ j y t j Such a formulation leads to N = 4r(s + 1) + s + T candidate regressors, so the approach is bound to generate N > T. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 7 / 37

Model selection in action Empirical example: UK real wages over the past century and a half. Objective: show that all aspects must be modelled jointly for a coherent economic model, including all substantively relevant variables, any dynamics, outliers and breaks, and non-linearities. 1 Explore non-linearities by undertaking model selection for several non-linear functions, retaining linearity in parameters. 2 Semi-automated approach: attempt to encompass selected model with a non-linear wage-price spiral term. 3 Encompassing 4 Exogeneity 5 Extended data set to forecast the last 7 years of real wages. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 8 / 37

Wages and prices 6 4 2 0 0.3 0.2 0.1 0.0 w p w p 4.5 4.0 3.5 3.0 1900 1950 2000 1900 1950 2000 w p 0.15 (w p) 0.10 0.05-0.1-0.2 1900 1950 2000 0.00 1900 1950 2000 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 9 / 37

Productivity, ULCs and unemployment 3.0 y l 0.3 ulc p 2.5 0.2 2.0 0.1 0.0 1.5-0.1 1.0 1900 1950 2000 0.10 0.15 Ur Ur 1900 1950 2000 0.10 0.05 0.05 0.00 1900 1950 2000-0.05 1900 1950 2000 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 10 / 37

Non-linearity test The Castle and Hendry (2010) non-linearity index test applied to a linear model of real wage growth, where the regressors include an intercept, (w p) t i and (ulc p) t i for i = 1, 2 and (y l) t j, U r,t j and p t j for j = 0, 1, 2. The test is significant at p = 0.006 with F(36, 91) = 1.95. Castle and Hendry (2009) also found the index test to be significant. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 11 / 37

Wage-price spirals Castle and Hendry (2009) found evidence for a non-linear real wage reaction to inflation: f t = 1 1 + 1000( p t ) 2. 0.0-0.1-0.2-0.3 fna -0.4-0.5-0.6-0.7-0.8-0.9-0.15-0.10-0.05 0.00 0.05 0.10 0.15 0.20 0.25 p Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 12 / 37

The extended and updated CH model (w p) t = 0.010 (0.002) + 0.649 (0.126) (ft pt ) + 0.384 (0.045) (y l) t + 0.159 (0.048) (y l) t 2 0.063 (ulc p) t 2 0.129 2Ur,t 1 + 0.029I1918 + 0.139 (0.010) (0.044) (0.013) (0.013) I1940 + 0.032 (0.006) (I1942 + I1943 I1944 I1945 ) 0.041 (0.009) (I1975 + I1977 ) (1) R 2 = 0.733; σ = 1.24%; SIC = 5.66; χ 2 nd (2) = 2.21; F ar (2, 130) = 0.766; F arch (1, 140) = 0.109; F het (13, 126) = 0.794; F reset (2, 130) = 0.106; F chow (7, 132) = 1.354; T = 1864 2004. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 13 / 37

The extended and updated CH model (w p) t = 0.010 (0.002) + 0.649 (0.126) (ft pt ) + 0.384 (0.045) (y l) t + 0.159 (0.048) (y l) t 2 0.063 (ulc p) t 2 0.129 2Ur,t 1 + 0.029I1918 + 0.139 (0.010) (0.044) (0.013) (0.013) I1940 + 0.032 (0.006) (I1942 + I1943 I1944 I1945 ) 0.041 (0.009) (I1975 + I1977 ) (1) R 2 = 0.733; σ = 1.24%; SIC = 5.66; χ 2 nd (2) = 2.21; F ar (2, 130) = 0.766; F arch (1, 140) = 0.109; F het (13, 126) = 0.794; F reset (2, 130) = 0.106; F chow (7, 132) = 1.354; T = 1864 2004. Update is close to original despite data revisions, and is relatively constant over the Great Recession. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 13 / 37

Model fit, residuals and forecast errors 0.15 (w p) Fitted a 2 scaled residuals forecast errors b 0.10 1 0.05 0 0.00 0.050 0.025-1 -2 1900 1950 2000 1900 1950 2000 1-step forecasts (w p) Residual density N(0,1) c 0.4 d 0.3 0.000 0.2 0.1 2005 2010-3 -2-1 0 1 2 3 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 14 / 37

An approximating model f t is a variant of an LSTAR in π 2 t where π t = 100 p t (annual inflation measured as a percentage), given by (scaling to the same mean and range as f t ): Lp t = 2 ( 1 + exp( γπ 2 t ) ) 1 2 so the approximation in the Taylor expansion becomes: α 1 p t + α 2 ( p t ) 3 + α 3 ( p t ) 4 Also included the most significant non-linear function of the other regressors, U 2 r,t. Selection at 1%. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 15 / 37

An approximating model Wage-price spiral as an LSTAR approximation (w p) t = 0.017 (0.003) + 0.314 (0.050) (y l) t + 0.184 (0.055) (y l) t 2 0.060 (0.013) (ulc p) t 2 0.166Ur,t + 2.59 (0.042) (0.80) U2 r,t 0.096 2Ur,t + 6.60 (0.050) (1.63) ( pt ) 3 17.7 (5.44) ( pt ) 4 0.186 pt 0.120 (0.045) (0.03) 2 p t 1 + 0.148I1940 0.044I1944 0.052 (0.013) (0.013) 0.038 (0.013) I1977 (0.013) I1945 R 2 = 0.747; σ = 1.23%; SIC = 5.55; (2) χ 2 nd (2) = 0.88; F ar (2, 124) = 0.63; F arch (1, 139) = 0.18; F het (19, 117) = 1.12; F chow (7, 126) = 1.61; T = 1864 2004. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 16 / 37

An approximating model Both non-linear terms in inflation are highly significant, and the fit and mis-specification tests are similar to (1). I 1922, I 1939 and I 1942 eliminated by U 2 r,t, ( p t ) 3 and ( p t ) 4. U r,t is intrinsically positive, so combined term, 0.18U r,t (1 13.7U r,t ), is negative till the unemployment rate exceeds 7.25% then becomes positive. Such an effect could represent movements along the marginal product curve, raising real wages of those still employed as employment fell, from more capital per worker and the unemployment of less productive workers. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 17 / 37

A nesting model (w p) t = 0.015 (0.003) + 0.348 (0.047) (y l) t + 0.204 (0.053) (y l) t 2 0.061 (0.011) (ulc p) t 2 0.157Ur,t + 2.56 (0.039) (0.79) U2 r,t 0.166 2Ur,t + 0.625(ft pt) 0.131 (0.050) (0.14) + 0.138I1940 0.042I1944 0.045I1945 0.046 (0.013) (0.013) (0.013) (0.012) R 2 = 0.747; σ = 1.22%; SIC = 5.61; T = 1864 2004; χ 2 nd (2) = 0.54; F ar (2, 126) = 0.96; F arch (1, 139) = 0.06; F het (15, 121) = 1.26; F reset (2, 126) = 0.28; F chow (7, 128) = 1.09. (0.031) 2 p t I1977 (3) The wage-price spiral term is not sufficient to model all non-linearity, but does explain that for impact of inflation on real-wage growth. Some restricted dummies drop out. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 18 / 37

Model fit, residuals and forecast errors 0.15 (w p) ^ (w p) a scaled residuals forecast errors 2 b 0.10 0.05 0 0.00-2 0.5 0.4 1900 1950 2000 Residual density N(0,1) c 1.0 0.5 1900 1950 2000 Residual correlogram d 0.3 0.2 0.0 0.1-0.5-4 -3-2 -1 0 1 2 3 0 5 10 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 19 / 37

An LSTAR model Replacing f t p t by Lp p t, non-linear estimation leads to γ = 0.06. Lp generates almost identical behaviour to f t (correlation 0.96). (w p) t = 0.018 (0.003) + 0.308 (0.049) (y l) t + 0.206 (0.053) (y l) t 2 0.074 (0.013) (ulc p) t 2 0.183Ur,t + 2.64 (0.042) + 0.822 (0.23) pt (0.80) U2 r,t 0.152 ( 1 + exp (0.049) 2Ur,t ( 0.059 (0.023) π2 )) 1 1.02 + 0.140I1940 0.045I1944 0.048I1945 0.043 (0.013) (0.013) (0.013) (0.013) (0.24) 2 p t 0.907 (0.24) pt 1 R 2 = 0.752; σ = 1.22%; SIC = 5.60; T = 1864 2004; χ 2 nd (2) = 0.31; F ar (2, 124) = 1.26; F arch (1, 139) = 0.14; F het (15, 121) = 1.81 ; F chow (7, 126) = 1.31. I1977 (4) Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 20 / 37

f and Lp f 0.00 a Lp 0.00 b -0.25-0.25-0.50-0.50-0.75-0.75-0.2-0.1 0.0 0.1 0.2 f 0.00 t -0.25 p -0.2-0.1 0.0 0.1 0.2 Lp 0.00 t c -0.25 d p -0.50-0.50-0.75-0.75 1900 1950 2000 1900 1950 2000 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 21 / 37

Encompassing Neither model (4) and (2) encompasses the other: Test Model 1 vs. Model 2 Model 2 vs. Model 1 Cox N(0,1) 4.66 7.25 Joint Model F(2,125) = 4.08 F(1,125) = 10.9 Neither Lp t p is significant if added to (3), nor f t p t when added to (4), so they are close substitutes. Despite p t entering ( w t p t ), real wages are primarily determined by forces different from nominal prices, consistent with the Classical dichotomy : in particular, the impact of p t on real wages is zero at high inflation. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 22 / 37

Alternative modelling of regime change Re-estimated model proposed by Nielsen (2009) (w p) t = 0.006 (0.002) + 0.882 (0.126) (f p) t + 0.297 (0.045) (y l) t 0.072 (ulc p) t 2 0.148 (0.013) (0.045) 2Ur,t ( ) +0.0003 I {1860 1913} U 1 r (0.0001) t ( ) +0.0003 I {1860 1913} U 1 r (0.00006) t 2 ( I{1947 2011} log (U r ) ) t 0.031 (0.008) 0.004 (0.0009) ( I{1947 2011} log (U r ) ) t 1 +0.036 (0.012) I1918t + 0.146I1940t + 0.039 I WWIIt 0.037 (0.012) (0.006) (0.008) I7577t R 2 = 0.783; σ = 1.13%; SC = 5.77; T = 1864 2004; χ 2 nd (2) = 0.53; F ar (2, 126) = 0.089; F arch (1, 139) = 0.003; F het (19, 119) = 1.093; F reset (2, 126) = 2.228; F chow (7, 128) = 1.218. (5) Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 23 / 37

Alternative modelling of regime change Log-likelihood for equation augmenting (5) by additional regressors from (3) and (1) is 446.4, and for (3) is 428.0, so F(4, 123) = 6.38. The regime-shift variables matter, and remain relevant over the great recession, as curtailing their influence to 2004 leads to marked deterioration in RMSFE. Neither model encompasses the other, as a test of those additional regressors yields F(5, 123) = 4.42. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 24 / 37

IIS and SIS Extension of IIS to step-indicator saturation (SIS): adding a complete set of step indicators S 1 = { 1 {t j}, j = 1,..., T }, where 1 {t j} = 1 for observations up to j, and zero otherwise. Step indicators are the cumulation of impulse indicators up to each next observation. IIS: Impulses 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0... SIS: Step Shifts 1 1 1 1 0 1 1 1 0 0 1 1 0 0 0 1 SIS has the correct null retention frequency in constant conditional models for a nominal test size of α. The approximate alternative retention-frequency function has been derived analytically for simple models, and shows higher probabilities of retaining location shifts. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 25 / 37

Illustrating SIS when no location shifts Split-sample search by SIS at 1%. Block 1 1.0 0.5 Indicators included initially 1.0 0.5 Indicators retained Selected model: actual and fitted 12.5 actual fitted 10.0 0 50 100 0 50 100 0 50 100 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 26 / 37

Illustrating SIS when no location shifts Split-sample search by SIS at 1%. Indicators included initially Indicators retained 1.0 1.0 Selected model: actual and fitted actual fitted 12.5 Block 1 0.5 0.5 10.0 Block 2 0 50 100 1.0 0.5 0 50 100 0 50 100 1.0 12.5 0.5 10.0 0 50 100 0.0 0.5 1.0 0 50 100 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 26 / 37

Illustrating SIS when no location shifts 1.0 Indicators included initially 1.0 Indicators retained Selected model: actual and fitted actual fitted 12.5 Block 1 0.5 0.5 10.0 Block 2 0 50 100 1.0 0.5 0 50 100 0 50 100 1.0 12.5 0.5 10.0 0 50 100 1.0 0.0 0.5 1.0 0 50 100 1.0 12.5 Final 0.5 0.5 10.0 0 50 100 0 50 100 0 50 100 T = 100, and no shifts, retains 2 significant steps, so lose 2 degrees of freedom but could be combined to one dummy. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 26 / 37

SIS for a single location shift Split-sample search in SIS when a shift occurs. Block 1 Indicators included initially Indicators retained 1.0 1.0 0.5 0.5 Selected model: actual and fitted 15 actual fitted 10 5 Block 2 Final 0 50 100 1.0 0.5 0 50 100 1.0 0.5 0 50 100 1.0 0.5 0 50 100 1.0 0.5 0 0 50 100 15 10 5 0 0 50 100 15 10 5 0 50 100 0 50 100 0 0 50 100 Initially retains last step as mean shifts down, then finds location shift, so eliminates the now redundant indicator: just one step needed. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 27 / 37

Encompassing model Checking robustness by applying SIS (w p) t = 0.030 (0.003) + 0.354 (0.042) (y l) t + 0.116 (0.034) 2 (y l) t 1 0.179 (0.028) (w p y + l µ) t 2 0.178Ur,t + 2.68 (0.034) (0.68) U2 r,t 0.13 2Ur,t + 0.711 (0.045) (0.012) (ft pt ) 0.130 0.145S1939 + 0.176S1940 0.058S1941 0.024 (0.011) (0.015) (0.011) (0.008) 0.036I1916 + 0.027 (0.011) (0.006) (I1942 + I1943 I1944 I1945 ) 0.044 R 2 = 0.820; σ = 1.04%; SIC = 5.85; (0.029) 2 p t 1 (S2011 S1946) ur,t I1977 (6) (0.011) χ 2 nd (2) = 2.26; F ar (2, 123) = 0.39; F arch (1, 139) = 0.49; F reset (2, 124) = 2.28; F het (20, 116) = 0.82; F chow (7, 125) = 0.95; T = 1864 2004. (w p y + l) replaces (ulc p) adjusted for changes in hours ( µ is sample mean of (w p y + l)). u r,t = log(u r,t ) (e.g.) S 1939 is step indicator: unity till 1939 and zero thereafter. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 28 / 37

Encompassing model 0.15 (w p) a ^ (w p) 2 scaled residuals forecast error b 0.10 1 0.05 0 0.00 0.050 0.025-1 -2 1900 1950 2000 1900 1950 2000 1-step forecasts (w p) Residual density N(0,1) c 0.4 d 0.3 0.000 0.2 0.1-0.025 2005 2010-3 -2-1 0 1 2 3 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 29 / 37

Testing exogeneity (6) encompasses the previous models All mis-specification tests are insignificant Most variables in common with (3) have similar coefficients, other than a stronger and more rapid feedback of almost 0.18 from the previous labour share, and replacing (y l) t 2 by 2 (y l) t 1, as well as switching from pure impulse dummies to a mixture of steps and impulses. Two of the variables from (5) are also retained, so an interaction of a step shift with a variable matters as well. The main role of the step indicators is explaining the much higher average growth rate of real wages post war (1.8% p.a., versus 0.7% p.a. pre-1945), even though (y l) is included and displays a similar pattern. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 30 / 37

Castle Table: (Oxford) IIS super-exogeneity Semi-automatic Non-linear tests Model of (3) Selection and SIS tests of (6). ISF 2013 31 / 37 Testing exogeneity IIS/SIS is used to test exogeneity of the conditioning variables, Hendry and Santos (2010). Under null of super exogeneity, parameters in the conditional model are invariant to shifts in the marginal models, so indicators in the latter should not enter the former. A VAR in w p, y l, p and U r was selected with IIS/SIS, and additional impulse indicators in the 3 marginal models were then tested for significance in (3)/(6). Super exogeneity tests Variable null distribution IIS test statistic null distribution SIS test statistic (y l) t F(11,117) 1.16 F(2,123) 0.77 p t F(11,117) 1.22 F(7,118) 1.87 U r,t F(9,118) 1.05 F(14,111) 1.37 Joint F(16,112) 1.22 F(20,105) 1.41

Forecasts for real wages 4.85 Equation (3), No IC 4.85 Equation (3) with IC 4.80 4.80 4.75 4.75 4.85 2005 2010 Equation (5), no IC 4.85 2005 2010 Equation (5) with IC 4.80 4.80 4.75 4.75 4.85 2005 2010 Equation (6), no IC 4.85 2005 2010 Equation (6) with IC 4.80 4.80 4.75 4.75 2005 2010 2005 2010 Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 32 / 37

Forecasting RMSFEs of forecasts of (w p) with and without intercept corrections Equation σ No IC IC (3) 1.22% 1.31% 1.25% (5) 1.13% 1.23% 1.04% (6) 1.04% 1.05% 1.00% RW 2.23% 1.57% 1.54% VAR 1.67% 2.37% 1.54% Table: RMSFEs of forecasts of (w p) T+h with and without intercept corrections, with in-sample equation standard error for comparison. IC is average residual over 2003 2004. Most complicated model has smallest RMSFE. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 33 / 37

Conclusions Joint modelling of dynamics, location shifts, relevant variables and non-linearities essential. Automatic model selection despite N > T seems a viable approach to tackling all complications jointly. Wage-price spiral adds a unit root to the wage-price process. Can be approximated in several ways. Real wages are primarily determined by forces different from nominal prices, consistent with the Classical dichotomy. A general polynomial led to an additional non-linearity in unemployment; real wages rose with unemployment beyond about 7.25%, probably rising marginal productivity rather than wage bargaining. Consistent with involuntary unemployment, as no evidence of any reverse relation of high real wages causing unemployment was found in Hendry (2001). Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 34 / 37

Conclusions Empirical evidence for non-linear adjustments of real wages to inflation. Reaction based on exogenous variable. IIS did not preclude finding non-linearites, and non-linearities removed indicators found in linear specifications. Not removing the large outliers could hide the presence of other variables, including the non-linearities. Most complicated model produced the smallest 1-step forecast errors rebuts parsimony. All forecasts benefitted from intercept corrections setting their forecasts back on track at the forecast origin. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 35 / 37

References I Castle, J. L., J. A. Doornik, and D. F. Hendry (2011). Evaluating automatic model selection. Journal of Time Series Econometrics 3 (1), DOI: 10.2202/1941 1928.1097. Castle, J. L. and D. F. Hendry (2009). The long-run determinants of UK wages, 1860 2004. Journal of Macroeconomics 31, 5 28. Castle, J. L. and D. F. Hendry (2010). A low-dimension portmanteau test for non-linearity. Journal of Econometrics 158(2), 231 245. Castle, J. L. and D. F. Hendry (2011). Automatic selection of non-linear models. In L. Wang, H. Garnier, and T. Jackman (Eds.), System Identification, Environmental Modelling and Control, pp. 229 250. New York: Springer. Doornik, J. A. (2009a). Autometrics. In J. L. Castle and N. Shephard (Eds.), The Methodology and Practice of Econometrics: A Festschrift in Honour of David F. Hendry, pp. 88 121. Oxford: Oxford University Press. Doornik, J. A. (2009b). Econometric model selection with more variables than observations. Working paper, Economics Department, University of Oxford. Granger, C. W. J. and T. Teräsvirta (1993). Modelling Nonlinear Economic Relationships. Oxford: Oxford University Press. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 36 / 37

References II Hendry, D. F. (2001). Modelling UK inflation, 1875 1991. Journal of Applied Econometrics 16, 255 275. Hendry, D. F. (2009). The methodology of empirical econometric modeling: Applied econometrics through the looking-glass. In T. C. Mills and K. D. Patterson (Eds.), Palgrave Handbook of Econometrics, pp. 3 67. Basingstoke: Palgrave MacMillan. Hendry, D. F. and S. Johansen (2013). Model discovery and Trygve Haavelmo s legacy. Econometric Theory, forthcoming. Hendry, D. F. and H.-M. Krolzig (2005). The properties of automatic Gets modelling. Economic Journal 115, C32 C61. Hendry, D. F. and C. Santos (2010). An automatic test of super exogeneity. In M. W. Watson, T. Bollerslev, and J. Russell (Eds.), Volatility and Time Series Econometrics, pp. 164 193. Oxford: Oxford University Press. Nielsen, H. B. (2009). Comment on the long-run determinants of UK wages, 1860 2004. Journal of Macroeconomics 31, 29 34. White, H. (1980). A heteroskedastic-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48, 817 838. Castle (Oxford) Semi-automatic Non-linear Model Selection ISF 2013 37 / 37