An Example Of Potential Pitfalls Associated With The Use And Abuse Of Default Parameter Settings Ric D. Herbert 1 Peter J. Stemp 2 1 Faculty of Science and Information Technology, The University of Newcastle, Ourimbah, NSW, Australia 2 Department of Economics, Monash University, Clayton East, Vic, Australia 15th International Conference on Computing in Economics and Finance, Sydney Australia, July 2009
Table of contents 1 2 3 4 5
This paper is part of a project we have been working on about the computational appropriateness of solution methods for dynamic macroeconomic models. There can be a temptation to rely on default settings in proprietary software to derive solutions to the model. In this paper we show that, for the solution of non-linear dynamic models, this approach can be inappropriate.
Method We consider versions of a simple model with stable and unstable eigenvalues, so that any dynamic solution requires the calculation of appropriate jumps in endogenous variables. One version of the model, expressed in levels, is highly non-linear. A second version of the model, expressed in logarithms, is linear. We start by showing the results that can be derived using the default settings of a typical solver. We show that this gives unsatisfactory results for the non-linear model; and then, show how the default settings and choice of solver can be adjusted to give acceptable results.
The linear model (expressed in logarithms) is given by the following set of equations: m p = α 1 y α 2 ṗ (1) y = β + (1 γ)n (2) ṅ = θ(δ γ)n (3) ẇ = η(n n) (4) The Greek symbols denote parameters with a positive value with 0 < γ < 1.
Model variables y = output; n = employment; n = full employment, assumed to be constant; p = price level; m = nominal money stock, assumed to be constant; and w = wage rate. is shocked by a change to the money stock.
Steady state Since this model is linear, it has a unique equilibrium which satisfies the following equations: m p = α 1 y (5) y = β + (1 γ)n (6) δ γn = w p (7) n = n (8)
Calibrating the model and eigenvalues α 1 = 1 α 2 = 0.5 β = 0 γ = 0.5 δ = log(0.5) θ = 1 η {0.2, 0.5, 10, 100} n = 1, m = 0.1, m 0 = 0 Eigenvalues η λ 1 λ 2 λ 3 0.2 2.3-0.50-0.34 0.5 2.3-0.41+0.51i -0.41-0.51i 10.0 2.1-0.32+3.0i -0.32-3.0i 100.0 2.0-0.26+9.9i -0.26-9.9i λ 1 indicates unstable pole as η increases eigenvalues become complex and have larger imaginary parts shock to m results in p jumping so economy moves on to the stable trajectory to new steady-state
Linear Model Eigenvalues for η [ 100, 100]. Eigenvalues as η varies Eigenvalues as η varies 10 10 λ 1 λ 2 λ 3 8 λ 1 λ 2 λ 3 5 6 4 Real part of eigenvalue 0 5 Imaginary part of eigenvalue 2 0 2 4 10 6 8 15 100 80 60 40 20 0 20 40 60 80 100 η 10 100 80 60 40 20 0 20 40 60 80 100 η Real part on LHS, imaginary part on RHS has a dominant real eigenvalue greater than zero for all the values of η. As η increases above 0.2 the model becomes more oscillatory.
p n w Analytic solution 0.39 Prices with varying η 1.1 Employment with varying η 1.5 Wages with varying η 0.395 1.52 0.4 1.08 1.54 0.405 1.06 1.56 0.41 1.58 0.415 1.04 1.6 0.42 1.62 0.425 1.02 1.64 0.43 1 1.66 0.435 1.68 0.44 Time 0.98 Time 1.7 Time The closed-form solution solved using standard matrix techniques Dynamic solution becomes more oscillatory as η increases
The non-linear model (expressed in levels) An equivalent model specification can be rewritten in levels as follows: ( ) PN (1 γ)α 1 1 α 2 Ṗ = P log (9) M ( ) P(1 γ)n γ θ Ṅ = N log (10) W ( ) η Ẇ = W log (11) N N The variables are as for the model above except that upper case letters denote levels.
Steady-states Some solutions to the non-linear model can be derived by using the following transformation P = exp(p) (12) N = exp(n) (13) W = exp(w) (14) There are multiple steady-states with the non-linear model and some are not economically meaningful. These include the steady-state: P = 0 (15) N = N (16) W = 0 (17)
Two Point Boundary Value Problem The computational solution to this model can be formulated as a a two point boundary value problem written as: ẋ = f(t, x) 0 < t < t f (18) g(x(0), x(t f )) = 0 (19) Where f(.) is either the linear model, or non-linear model and g(.) is obtained from the steady-state equation. For each initial conditions vector c, the model has a solution x(t) = x(t; c) which satisfies x(0; c) = c. Substituting into equation 19 gives: g(c, x(t f ; c)) = 0 (20) and the problem is to solve this set of non-linear equations.
Solving 2PBVP The common approaches to solving the problem are by shooting, finite differences or projection methods. Here we will focus on on a shooting method Shooting methods for this model require An ODE solver to generate x(t) = x(t; c) A searcher to solve g(c, x(t f ; c)) = 0 The focus of this paper is on the model solver and default parameters of the solver.
Previous Work The nonlinear model can only be solved by computational means. We focus on shooting algorithms and particularly forward (in time) and reverse shooting. In Stemp and Herbert (2006) we focussed on the oscillatory linear model (η = 100) as it may have similar dynamics to nonlinear models. We showed that with η = 100 the choice of the ODE solver has significant impact on the likely success of shooting algorithms. In Herbert and Stemp (2009) we focussed on the nonlinear model and showed that forward-shooting is likely to solve to the wrong solution for the nonlinear model but the right solution for the linear model.
Approach We use the reverse shooting algorithm and compare the solution to the closed-form solution (expressed in levels). We use Matlab as the software environment. We consider 3 candidate ODE solvers which could typically be used for this model. We focus of η = 0.2 so that all eigenvalues are real.
ODE Solvers Runge-Kutta (Fehlberg method, Dormand-Prince 4/5 formula) A good robust single-step solver Solver of first choice for problem like this. (default!) Matlab ode45() function Adams-Bashforth-Moulton A multi-step solver Not normally used for a model like this as model is not computationally expensive to evaluate. Matlab ode113() function Stiff This model is not obviously stiff. (Largest eigenvalue about 4 times smallest) But stiffness could be the cause of the problems. Matlab ode115s() function
Default RK solver linear model solution Prices [!=0.2] Employment [!=0.2] 0.675 2.95 0.67 0.665 0.66 0.655 0.65 0.645 Time 2.9 2.85 2.8 2.75 2.7 Time 0.205 Wages [!=0.2] Phase [!=0.2] 0.2 0.21 0.195 0.19 0.185 0.18 Time Employment 0.2 0.19 0.18 3 0.68 2.8 0.66 Wages 2.6 0.64 Prices
Default RK solver nonlinear model solution Prices [!=0.2] Employment [!=0.2] 0.675 3 0.67 0.665 0.66 0.655 0.65 0.645 Time 2.9 2.8 2.7 2.6 Time 0.205 Wages [!=0.2] Phase [!=0.2] 0.2 0.21 0.195 0.19 0.185 0.18 Time Employment 0.2 0.19 0.18 3 0.68 2.8 0.66 Wages 2.6 0.64 Prices
Norm results for all methods: non-linear model when η = 0.2. (Root Mean Square Error) Method RMSE All RMSE P RMSE N RMSE W RK Solver 0.007744 0.001996 0.021559 0.000506 RK Solver 0.000478 0.000936 0.001303 8.60E-06 ABM Solver 0.000267 0.000521 0.000743 5.58E-06 ABM Solver 0.000826 0.001622 0.002254 1.45E-05 ABM Solver 0.000811 0.001591 0.002212 1.42E-05 ABM Solver * 0.000133 0.000248 0.000364 5.03E-06 Stiff Solver 0.125903 0.034000 0.305671 0.007573 Stiff Solver 0.000762 0.001470 0.002083 1.61E-05 tolerances half the defaults on solver (i.e. 10 6 ) tolerances half the defaults on solver and refine (dense output) * max solver step size of 0.01
Linear model with ABM limited step-size solver Prices [!=0.2] Employment [!=0.2] 0.675 2.95 0.67 0.665 0.66 0.655 0.65 0.645 Time 2.9 2.85 2.8 2.75 2.7 Time 0.205 Wages [!=0.2] Phase [!=0.2] 0.2 0.21 0.195 0.19 0.185 0.18 Time Employment 0.2 0.19 0.18 3 0.68 2.8 0.66 Wages 2.6 0.64 Prices
Non-linear model with ABM limited step-size solver Prices [!=0.2] Employment [!=0.2] 0.675 2.95 0.67 0.665 0.66 0.655 0.65 0.645 Time 2.9 2.85 2.8 2.75 2.7 Time 0.205 Wages [!=0.2] Phase [!=0.2] 0.2 0.21 0.195 0.19 0.185 0.18 Time Employment 0.2 0.19 0.18 3 0.68 2.8 0.66 Wages 2.6 0.64 Prices
Comments The standard (default) software solver did not perform well. (Runge-Kutta-Fehlberg). Stiffness does not seem to be the issue. None of the solvers with default parameter settings performed well. Usual practices of reducing tolerances and using dense output also did not perform well. The technique which worked best is the least preferred practice of reducing step-size.
Take Home Message Comment We are a team that has extensive software development and model solving experience. We have used best-practice software and best-practice algorithms. We have shown that these can have problems solving a basically trivial model. What about the model solutions we see out there for complex models using simple software? Garbage in garbage out is the old software developers saying.
References Herbert, Ric D. and Peter J. Stemp, Solving a non-linear model: the importance of model specification for deriving a suitable solution, Mathematics and Computers in Simulation, 2009, vol. 79, no. 9, pp. 2847 2855. Stemp, Peter J. and Ric D. Herbert, Solving non-linear models with saddle-path instabilities,computational Economics, 2006, vol. 28, no. 2, pp. 211 231.