Lecture 7: Numerical Tools

Transcription

1 Lecture 7: Numerical Tools Fatih Guvenen January 10, 2016 Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

2 Overview Three Steps: V (k, z) =max c,k 0 apple u(c)+ Z c + k 0 =(1 + r)k + z z 0 = z +. V (k 0, z 0 )f (z 0 z)dz 0 1 How to perform the maximization in the Bellman objective? (Constrained optimization) 2 As part of step (1), evaluate E(V (k 0, z 0 ) z) repeatedly. Two components: 1 Interpolation: Maybe required twice: Interpolate in the k 0 direction. Also interpolate in the z 0 direction, if f (z 0 z = z j ) is continuous. 1 Integration: Again, if f (z 0 z = z j ) is continuous, we need to integrate. One option will be to treat it as discrete (saves time and headaches; not always feasible). Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

3 ROOT FINDING

4 Introduction I Let f : R N! R N. Solve f (x) =0. Depending on f, there may be zero, one, or multiple solutions. I I will start with the root finding problem in 1-dimension (N = 1). I This is a special case where we can guarantee that we are bracketing a zero. I In higher dimensional problems, no method can (generically) guarantee that you are bracketing a zero. I In higher dimensions, I often convert zero finding into a minimization problem. More on this in a moment. Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

5 Bisection, Secant and False Position Methods Many zero finding methods begin with two points that bracket a zero: f (a) f (b) < 0 I Bisection: 1 First, evaluate f ( a+b 2 ). For x = a, b 2 if f ( a+b a+b 2 ) f (x) > 0, replace x with 2. Go back to step 1 (1). Bisection is relatively slow (linear convergence) but converges for sure. I Secant and False position methods: Idea: Treat f ( ) as if it is linear near the zero. Given f (a) and f (b), solve for the zero of the line that connects these two points. Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

6 Secant and False Position Methods The two methods are very similar but differ in whether they lean towards speed (secant) or robustness (false position) (see fig 9.2 in Numerical Recipes for a comparison) I Secant: take the two most recently evaluated points. Secant method is very aggressive because it always throws out the oldest evaluation, which yields superlinear convergence: lim k!1 k+1 K I False position: take two most recent points that bracket zero. Typically slower than Secant because it sometimes keeps an older evaluation in the interest of bracketing a zero (hard to determine exact convergence rate). Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

7 What to Use in Practice? I None of the above! These simple ideas are often building blocks of more complex methods that you should use in practice. But they are useful for understanding more complex methods. 1 If derivatives are (i) painfully expensive to compute or (ii) unreliable (we will see some examples) then use Brent s method 2 Else, use the Newton-Raphson method (when you know you are near a zero). I TIP: Convert zero finding into a minimization problem: if f (a) f (b) < 0 then the squared function f ( ) 2 will have a local minimum at the zero of f ( ). I Then one can use various methods for minimization (not surprisingly Brent and Newton s methods also have versions for optimization!) Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

8 Brent s method TIP: Brent s method is your best bet if you really want to avoid using derivatives. I It combines the speedier version of the Secant method with the reliability of bisection. I Basically, it takes two points (a, b) that bracket a zero, and a third one in between (which could be a+b 2 ) I It fits a quadratic (instead of the linear function in Secant) to these three points. This yields faster convergence. I However, it carefully checks that the points always bracket a zero and the algorithm converges nicely. If not, it reverts back to bisection for a while. Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

9 Newton-Raphson method f (x + )=f (x)+ f 0 (x)+higher order terms we will ignore... Setting f (x + )=0yields = f (x) f 0 (x). Therefore, begin with x 0 and update: f (x) = x t+1 x t ) x t+1 = x t f 0 (x) Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

10 Newton-Raphson method Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

11 Newton-Raphson method I Pros: I Quadratic convergence with each step! (That is, the number of significant digits doubles in every step!) Extends easily to multi-dimensional problems. I Cons: computation of derivatives can be very slow if done numerically (as opposed to having an analytical formula). Poor global convergence properties. If there is a local minimum near the zero, updating can overshoot to infinity. Use with maximum care!! I What to do? Use a slow but sure method early on (like bisection) but then once you are close to the root switch to Newton-Raphson. Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

12 Problematic Cases Figure: Global Divergence, Local Convergence Local Convergence Global Divergence 10 5 f(x) 0 Divergent region Divergent region Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

13 Newton s Fractals I Consider the equation: z 3 1 = 0. It has three roots: z 1 = 1, z 2 = i p 3/2, z 2 = 0.5 i p 3/2. I The three points line up on a unit circle in the complex plane, separated by 120 degrees. I Depending on the starting point, Newton s method can converge to one of three points or can bounce around with chaotic dynamics. I The whole set feature fractals (near the boundaries). Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

14 Newton s Fractals Figure: Original set Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

15 NUMERICAL DERIVATIVES

16 Numerical Differentiation I TIP: Underestimate numerical differentiation at your own risk! f 0 f (x + h) (x) =lim h!0 h f (x) I Take a small h, say and evaluate (1). Looks pretty straightforward, no? (Hint: No!) I Computing the equilibrium of an economy requires satisfying a set of conditions (market clearing, optimality of choices, etc.) that can be expressed as root finding problems. I Once you get near a zero, you often want to use a Newton based approach! requires numerical derivatives. I If f (x) is excess demand and x is the price, calculating f (x + h) and f (x) requires solving the entire model twice! I If you have a complicated model this can be very time consuming. (1) Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

17 Numerical Differentiation I How to choose h? Two issues: 1 Rounding error: As long as you use double precision variables not a major issue. 2 Truncation error: Crucial!! I If f (x) is excess demand, calculated with truncation error f, f 0 will have error ~ p f. I Because f m (machine precision: typically ~10 15 for DP), the best lower bound is p m. I Actual error will often be much larger, because f m. I I I So, to successfully clear mkts, you need to solve the decision rules precisely. But this is costly (and somewhat pointless) when your x is far away from x. Notice that in some cases taking a small h may amplify the truncation error leading to less precision in f 0. If you choose a termination condition for your program that is too small (e.g., f 0 < f /2), it may never converge! Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18

18 Numerical Differentiation I Two-sided derivatives: A better but slower approach: f 0 (x) =lim h!0 f (x + h) f (x h) 2h I This also requires two function evaluations. So why is it slower? Because often you need f (x) for other reasons, so you are already going to compute it. So two-sided derivatives require 2 additional evaluations compared to 1 for one-sided. I What s the benefit? For one-sided derivatives, truncation error is h. For two sided, it is h 2! Choose a smaller h than in one-sided case though: assuming you know the error in evaluating f (x), choose h 1/3 f. Fatih Guvenen Lecture 7: Numerical Tools January 10, / 18