George Alogoskoufis, Dynamic Macroeconomic Theory, 2015 Mathematical Annex 1 Ordinary Differential Equations In this mathematical annex, we define and analyze the solution of first and second order linear differential equations. Knowledge of the properties of differential equations is indispensable for analyzing problems in continuous time in dynamic macroeconomics. 1 We shall start with some definitions, and will then proceed to examine solution methods of first and second order differential equations, with both constant and variable coefficients. We shall finally briefly discuss the generalization of the solution methods to a system of n first order differential equations. A1.1 Definitions A differential equation is a mathematical equation derived from an unknown function of one or more variables, which connects the function itself, and its derivatives of various degrees. While the solution of a simple equation or a system of equations, is to find a constant or a set of constants which satisfy these equations, the solution of a differential equation, or a system of differential equations, is to find functions, which, along with their derivatives, satisfy the differential equation or the system of differential equations. The equations we shall consider are all function of time t, which is assumed a continuous variable. For example, the solution to the differential equation, y (t) = dy (A1.1) dt = a is a function y(t), the first derivative of which with respect to time is equal to a. The general solution to this differential equation is the function, y(t) = at + c (A1.2) where c is an arbitrary constant. The properties and the solution methods of differential equations are presented in a number of textbooks on 1 mathematics for economists. See Chiang (1974) and Simon and Blume (1994) for two of the best such textbooks. For a more advanced treatment of differential equations, see Boyce and DiPrima (1977).
A particular solution can be found using a boundary condition. For example, if we know that y at t=0 is equal to y0 (the boundary condition), then a particular solution 0f (A1.2) is given by determining c as c= y0. Another example is the differential equation, d 2 y y (t) = (A1.3) dt = a 2 whose solution is, y(t) = a (A1.4) 2 t 2 + bt + c where b and c are two arbitrary constants. A solution of a differential equation is a function y(t), which, along with its derivatives, satisfies the differential equation. A general solution is the full set of solutions of a differential equation. A particular solution requires the determination of the arbitrary constant or constants of integration. Differential equations are classified by their order, which is none other than the order of the highest derivative that appears in the equation. For example, (A1.1) is a first-order differential equation, whereas (A1.3) is a second-order differential equation. A differential equation is linear if the unknown function y(t) and and its derivatives are linear. Otherwise it is nonlinear. A differential equation can be solved by a method which is known as separation of variables, if it can be written as the equation of a term which contains only y, to a term which contains only t. For example, the equation, g(y) y = f (t) (A1.5) can be written as, g(y)dy = f (t)dt (A1.6) The variables are separated, and the solution is, g(y)dy = f (t)dt + c (A1.7) where c is an arbitrary constant. Finally, a differential equation, 2
f (t, y) + g(t, y) dy dt = 0 (A1.8) which is equivalent to, f (t, y)dt + g(t, y)dy = 0 (A1.9) is called exact, if there is a function U(t,y) which satisfies, du U t dt + U y dy fdt + gdy (A1.10) Thus, a differential equation is exact if it constitutes exactly the total differential of a function. A1.2 First Order Linear Differential Equations First order linear differential equations are distinguished between equations with constant coefficients, and equations with variable coefficients. A1.2.1 Constant Coefficients A first order linear differential equation with constant coefficients takes the form, y (t) + ay(t) = b (A1.11) where a and b are given constants. In order to find the function y(t) which satisfies (A1.11), note that, d(e at y(t)) = ae at y(t) + e at y (t) = e at y (t) + ay(t) (A1.12) dt If the differential equation (A1.11) is multiplied by e at, its left hand side is an exact differential equation, i.e the total differential of a function with respect to t. e at is called the integrating factor. Multiplying both sides by dt, we get, d( e at y(t) ) = be at dt whose integral is, e at y(t) = e at b a + c where c is the constant of integration. 3
Multiplying both sides by e at we get, y(t) = b (A1.13) a + ce at as the family of functions which satisfy the differential equation (A1.11). This family is called the general solution of (A1.11). In order to determine the constant of integration c we need to know the value of the function at some point in time. For example, if we know that at the point in time t=0, y(0) = y 0 then we know that, y 0 = b, which implies. a + c c = y b 0 a The general solution of the differential equation (A1.11) which satisfies y(0) = y 0 is, y(t) = y 0 e at + (1 e at ) b (A1.14) a = b a + (y b 0 a )e at In conclusion, in order to solve a linear first-order differential equation with constant coefficients, we multiply it by the integrating factor and take the integral. To calculate the constant of integration we use the value of the function at some point. The point that we use is called an initial condition or a terminal condition, or, more generally, a boundary condition. A1.2.2 Variable Right Hand Side If the right hand side of (A1.11) is not constant, but a known function of time, the solution method is similar. We multiply by the integrating factor and take the integral. For example, in the differential equation, y (t) + ay(t) = be λt (A1.15) multiplying by the integrating factor and separating the variables, we get, ( ) = be (a+λ)t dt d e at y (A1.16) Taking the integral of both sides of (A1.16), b e at y(t) = (A1.17) a + λ e(a+λ)t + c 4
Dividing both sides by the integrating factor, we get the solution, b y(t) = (A1.18) a + λ eλt + ce at (A1.18) is the family of functions satisfying(a1.15). The unknown constant c can again be determined by a boundary condition. A1.2.3 Variable Coefficients The general form of a first order linear differential equation is, y (t) + a(t)y(t) = b(t) (A1.19) where a(t) and b(t) are known functions, and we seek the function y(t). The function b(t) is often called a forcing term, and is considered exogenous. The integrating factor in this case is, e a(t ) dt as, d(y(t)e a(t ) dt ) = e a(t ) dt y (t) + a(t)y(t) (A1.20) dt Thus, multiplying (A1.19) by this integrating factor, and taking the integral, we get, y(t)e a(t ) dt = b(t)e a(t ) dt dt + c (A1.21) Dividing (A1.21) by the integrating factor, we finally get, y(t) = e a(t ) dt b(t)e a(t ) dt dt + e a(t ) dt c (A1.22) where c is the constant of integration.(a1.22) is the general solution of (A1.19). A particular solution requires a boundary condition that will determine the unknown constant c. Note, that it is not advised to apply the solution (A1.22) to any equation. It is simpler in many cases to multiply by the integrating factor and take the integral. A1.2.4 Homogeneous and Non-homogeneous Differential Equations If b=0 in (A1.11), the differential equation to be solved is called homogeneous. Otherwise, it is called non-homogeneous. The general solution of a differential equation consists of the sum of the general solution to the relevant homogeneous differential equation, i.e setting b=0 and solving, and a particular solution to the general equation (A1.11). 5
For example, the general solution to the homogeneous equation, y (t) + ay(t) = 0 (A1.23) derived from (A1.11) is, y(t) = ce at (A1.24) A particular solution, setting for example y (t) = 0, is, y* = b a (A1.25) Consequently, the general solution of the non-homogeneous differential equation (A1.11) is the sum of (A1.24) and (A1.25), i.e the general solution of the relevant homogeneous differential equation, sometimes called the complementary function, plus the particular solution for a constant y, otherwise known as the particular integral. The general solution is thus given by, y(t) = b (A1.26) a + (y b 0 a )e at This methodology is not as necessary for solving first order linear differential equations, but becomes very useful for differential equations of order higher than one, or systems of first order linear differential equations. In many economic applications, we are interested in the behavior of the solution of a differential equation as the independent variable, usually time, tends to infinity. The value to which the solution converges is referred to as a stationary state, or steady state, or equilibrium state. For example, from (A1.13), which is the general solution of (A1.11), for a > 0, we get, b lim y(t) = lim t t a + ce at = b a The particular integral of the differential equation (A1.11) can therefore be interpreted economically as the equilibrium state, or the steady state, which is the state towards which the variable y converges as times goes to infinity. This equilibrium is called a stable node. It is a stable equilibrium if y is a predetermined variable and only changes gradually, as postulated by the law of motion (A1.26). Assume now that a<0. In this case, if the boundary condition y0 is different from the steady value y*, y(t) as determined by (A1.26). It moves to plus or minus infinity, further and further away from the steady state. The only case in which this does not happen is when the boundary condition y0 is equal to the steady state value y*=b/a. Then y remains constant at y*. However, this is an unstable equilibrium, called a saddle point. There is only one adjustment path that leads to, and this is for y 6
to jump immediately to the steady state. If y is a non predetermined variable, for example a financial variable, or any variable that can change immediately and not gradually, then the economy can jump immediately to the steady state. A1.3 Second Order Linear Differential Equations A second order linear differential equation has the form, y (t) + a(t) y (t) + b(t)y(t) = h(t) (A1.26) where a(t), b(t), h(t) are known functions, and what is sought is the function y(t). The forcing term in this case is the function h(t). (A1.26) is referred to as the complete equation and in non-homogeneous. Related to (A1.26) is a homogeneous differential equation, in which h(t)=0. y (t) + a(t) y (t) + b(t)y(t) = 0 (A1.27) which is sometimes called the reduced equation. The full equation is non-homogeneous, while the reduced equation is homogeneous. The reduced equation is of interest because of the following two theorems. Theorem 1: The general solution of the complete equation (A1.26) is the sum of any particular solution of the complete equation, and the general solution of the reduced equation (A1.27). Theorem 2: Any solution y(t) of the reduced equation (A1.27) on t 0 t t 1 can be expressed as a linear combination, y(t) = c 1 y 1 (t) + c 2 y 2 (t), t 0 t t 1 of any two particular solutions y 1, y 2 which are linearly independent. A1.3.1 Homogeneous Equations with Constant Coefficients We now examine the differential equation (A1.26), with constant coefficients, that is, a(t)=a, b(t)=b. We also assume that h(t)=0. The differential equation thus takes the form, y (t) + a y (t) + by(t) = 0 (A1.28) Inspired by the general solution of the first order linear differential equation with constant coefficients, we try the general solution, y(t) = ce rt with unknown constants c and r. This solution method is called the method of undetermined coefficients. This solution implies that, y (t) = rce rt και y (t) = r 2 ce rt 7
Substituting in (A1.28) we get, ce rt (r 2 + ar + b) = 0 (A1.29) For a non zero c, our trial solution satisfies (A1.28) only if r is a root of the second order equation, r 2 + ar + b = 0 (A1.30) Equation (A1.30) is called the characteristic equation of (A1.28). It has two roots, which can be found from, r 1,r 2 = a ± a2 4b (A1.31) 2 We distinguish between three cases: Case 1: a 2 > 4b In this case the roots are real and distinct. The general solution of (A1.28) takes the form, y(t) = c 1 e r 1t + c 2 e r 2t (A1.32) where r 1,r 2 are the roots of the characteristic equation (A1.30), and c 1,c 2 are arbitrary constants. Case 2: a 2 < 4b In the case the roots are a pair of complex conjugates, r 1,r 2 = a, 2 ± i 4b a 2 = α ± iβ 2 where, α = a 4b a2 και β =. 2 2 The general solution in this case is, y(t) = e αt (k 1 cosβt + k 2 sinβt) (A1.33) where k 1,k 2 are arbitrary constants. Case 3: a 2 = 4b In this case the two roots are the same, and equal to -a/2. One can show that the general solution of (A1.28) in this case, takes the form, 8
y(t) = c 1 e rt + c 2 te rt = e rt (c 1 + c 2 t) (A1.34) where r = a / 2 is the double root of the characteristic equation (A1.30), and c 1,c 2 are arbitrary constants. A1.3.2 Non Homogeneous Equations with Constant Coefficients We have already derived the solution of any homogeneous second order linear differential equation with constant coefficients. In order to find the solution of a non homogeneous equation, we need a particular solution of the complete equation. If the complete equation is of the form, y (t) + a y (t) + by(t) = h (A1.35) then a particular solution is the constant function, y* = h b The full solution is thus the sum of the general solution to the homogeneous equation, plus the particular solution to the complete equation. For differential equations with variable coefficients, more advanced methods, such as the method of variation of parameters can be utilized. A1.4 A Pair of First Order Linear Differential Equations We next examine a case with extensive applications in economics, a pair of first order linear differential equations of the form, x (t) = a 1 x(t) + y(t) + p(t) y (t) = a 2 x(t) + b 2 y(t) + g(t) (A1.36) where a 1,a 2,,b 2 are given constants, and p(t),g(t) are given functions. The solution of the system of differential equations (A1.36) will be two functions x(t) and y(t), that satisfy both differential equations. The homogeneous system that corresponds to (A1.36) is given by, x (t) = a 1 x(t) + y(t) y (t) = a 2 x(t) + b 2 y(t) (A1.37) 9
One solution method is the method of substitution. Substituting y(t) and its derivatives in the equation determining x(t), we end up with a second order linear differential equation that contains only x(t) and its derivatives. x (t) (a1 + b 2 ) x (t) + (a 1 b 2 a 2 )x(t) = 0 (A1.38) (A1.38) is a linear homogeneous second order differential equation, and can be solved using the method of undetermined coefficients. Its characteristic equation is given by, r 2 (a 1 + b 2 )r + (a 1 b 2 a 2 ) = 0 (A1.39) If the roots of (A1.39) are real and distinct, the solution of (A1.38) is given by, x(t) = c 1 e r 1t + c 2 e r 2t (A1.40) Solving the first equation of (A1.37) with respect to y(t), we get, y(t) = 1 x (t) a 1 x(t) Substituting the solution (A1.40) for x(t) and its first derivative, we get, ( ) y(t) = 1 (r 1 a 1 )c 1 e r 1t + (r 2 a 1 )c 2 e r 2t (A1.41) Consequently, the solution of the system (A1.37) consists of equations (A1.40) and (A1.41), if the roots of (A1.39) are real and distinct. We can solve the system in an analogous way if we have complex or repeated roots. However, there is a second and more direct solution method of the homogeneous system (A1.37). Our experience with first order differential equation, suggests that we use the pair, x(t) = Ae rt, y(t) = Be rt as particular solutions for (A1.37). Substituting those in (A1.37) we get, rae rt = a 1 Ae rt + Be rt rbe rt = a 2 Ae rt + b 2 Be rt (A1.42) Dividing both equations by e rt we can rewrite the system (A1.42) as, a 1 r A (A1.43) a 2 b 2 r B = 0 0 10
For (A1.43) to hold, the determinant of the matrix of coefficients must be zero. a 1 r = 0 (A1.44) b 2 r a 2 Calculating the determinant, we get a second order equation in r. r 2 (a 1 + b 2 )r + (a 1 b 2 a 2 ) = 0 (A1.45) which is referred to as the characteristic equation of the system (A1.37). (A1.45) is exactly the same as (A1.39), the equation we ended up using the method of substitution. The solutions of the characteristic equation (A1.45) are called the eigenvalues of the matrix of coefficients, a 1 a 2 b 2 The two roots are given by, r 1,r 2 = (a + b ) ± (a + 2 1 2 )2 4(a 1 b 2 a 2 ) (A1.46) 2 Note, for future use, that, r 1 + r 2 = a 1 + b 2 r 1 r 2 = a 1 b 2 a 2 (A1.47) If the roots are real, and r 1 r 2, then, the general solution of the homogeneous system (A1.37) is given by, x(t) = A 1 e r 1t + A 2 e r 2t y(t) = B 1 e r 1t + B 2 e r 2t (A1.48) where, A 1, A 2 are determined by boundary conditions, the roots are determined by (A1.46), and B 1, B 2 are determined by (A1.42) as, B 1 = r a 1 1 A 1 and B 2 = r a 2 1 A 2 (A1.49) The solution is exactly the same as (A1.40) and (A1.41). In the case of complex or repeated roots, the solution is analogous. 11
Having found the general solution to the homogeneous system (A1.37), it remains to find a particular solution in (A1.36), using for example the method of variation of parameters. For the special case, where p and g are constants, a special solution with constant x and y can be found, by solving the system of equations, a 1 x *+ y *+ p = 0 a 2 x *+b 2 y *+g = 0 (A1.50) Expressing (A1.50) in matrix form, and solving for x* and y*, one gets, a 1 x *, which implies, (A1.50 ) a 2 b 2 y * = p x * g y * = a 1 p a 2 b 2 g x* and y* can be regarded as steady state, or equilibrium points. Whether the system converges globally to equilibrium depends on whether both roots are real and smaller than zero. In this case the equilibrium is a fixed node. If both variables are predetermined, this is a stable equilibrium. Where we have a positive and a negative root, the equilibrium is called a saddle point. There is only a unique path that leads to this equilibrium, and this path is called the saddle path. The economy will converge to equilibrium if one variable is predetermined and the other non predetermined. The non predetermined variable will jump to the unique adjustment path leading to equilibrium. Technically, the negative root corresponds to the predetermined variable, for which we solve backwards, and the positive root corresponds to the non predetermined variable, for which we solve forward. Thus, a system with one predetermined and one non predetermined variable has an equilibrium (a saddle point) if the matrix of coefficients has one positive and one negative eigenvalue. A1.5 A System of n First Order Linear Differential Equations We finally turn to a more general case, with extensive applications in economics, a system of n first order linear differential equations of the form, 1 x 1(t) = a 11 x 1 (t) + a 12 x 2 (t) ++ a 1n x n (t) + g 1 (t) x 2(t) = a 21 x 1 (t) + a 22 x 2 (t) ++ a 2n x n (t) + g 2 (t)... x n(t) = a n1 x 1 (t) + a n2 x 2 (t) ++ a nn x n (t) + g n (t) 12
where x 1, x 2,x n are variables, a ij for i, j = 1,2,,n are given constant parameters, and g 1,g 2,g n are exogenous functions of time. In matrix form, this system can be written as, x 1(t) a 11 a 1n x 1 (t) g 1 (t) = " (A1.51) x + n(t) a n1 # a nn x n (t) g n (t) or, x (t) = Ax(t) + g(t) (A1.51 ) where bold letters denote vectors, and A is the coefficient matrix, which is assumed non singular. A1.5.1 Eigenvalues and Eigenvectors Before we proceed to discuss the solution of the system of differential equations (A1.51), it is worth delving a little more into linear algebra, and in particular the concepts of eigenvalues and eigenvectors. Let A be a square matrix, like the one multiplying the x s in the right hand side of (A1.51). An eigenvalue of A is a number ρ, which when subtracted from each of the diagonal elements of A converts A into a singular, i.e. non invertible, matrix. Subtracting a scalar ρ from each of the diagonal elements of A is equivalent to multiplying A by ρ times the identity matrix I. Therefore, ρ is an eigenvalue of A if and only if A-ρΙ is singular. Since a matrix is singular if its determinant is equal to zero, ρ is an eigenvalue of A if and only if, det A ρi = 0 For an n x n matrix A, the left hand side of (A1.52) is an n-th order polynomial in ρ, called the characteristic polynomial of A. An n-th order polynomial has at most n roots. Therefore, an n x n square matrix has at most n eigenvalues. It follows from the above that the diagonal entries of a diagonal matrix D are eigenvalues of D, and that a square matrix A is singular if and only if 0 is an eigenvalue of A. Recall from elementary linear algebra that a square matrix B is non singular, if and only if the only solution of Bx = 0 is x = 0. Conversely, B is singular if and only if the system Bx = 0 has a non zero solution. 13
The fact that A-ρΙ is singular when ρ is an eigenvalue of A, means that the system of equations (AρΙ)v = 0 has a solution other than v = 0. 2 When ρ is an eigenvalue of A, a non-zero vector v such that (A-ρΙ)v = 0, is called a (right) eigenvector of A, corresponding to the eigenvalue ρ. Thus, eigenvectors are non-zero vectors v which satisfy, (A - ρι)v = 0, Av - ριv = 0, Av = ρv All these three statements are equivalent. A1.5.2 Solving the n-th Order System of Linear Differential Equations We now turn to the solution of the n-th order system of linear differential equations represented by (A1.51). The general solution of the non-homogeneous system of differential equations (A1.51) will be the sum of the general solution of the relevant homogeneous system of differential equations, plus the particular solution for constant x s. We shall concentrate on the solution of the homogeneous system, x 1(t) a 11 a 1n x 1 (t) = " (A1.52) x n(t) a n1 # a nn x n (t) or simply x = Ax, where x is the column vector on the right hand side of (A1.52). Assume first that A is a diagonal matrix, for which aij=0 for i j. Then, (A1.52) becomes a system of n independent self contained equations, of the form, x i(t) = a ii x i (t) We thus have a system of independent first order linear differential equations which can be solved, one by one, as, x i (t) = c i e a iit If the off diagonal elements aij differ from zero, so that the equations are linked to each other, then we can use the eigenvalues and eigenvectors of the coefficient matrix of (A1.52) to transform it to a system of n, or less, independent equations. We can use the eigenvalues and eigenvectors of A to a transform the system to a system which has a diagonal coefficient matrix. Let us assume that A has n distinct real eigenvalues ρ1, ρ2, ρn, with corresponding eigenvectors v1, v2, vn. It then follows from the definition of eigenvalues and eigenvectors that, 2 We follow the practice of using bold lowercase letters to denote vectors. 14
Av i = ρ i v i, i=1,2,, n (A1.53) Let P be the n x n matrix whose columns are these n eigenvectors. Thus, P is defined as, [ v n ] P = v 1 (A1.54) The system of equations (A1.53) can be written as, ρ 1 0 AP = PJ, where, J = (A1.55) 0 ρ n Since eigenvectors for distinct eigenvalues are linearly independent, P is non singular and therefore invertible. We can write, P 1 AP = J (A1.56) Thus, we can use (A1.56) to transform the system (A1.52), defined in the variables x, to a system in the variables y=p -1 x, which means that x=py. It follows that, y = P 1 x = P 1 Ax = P 1 APy = Jy (A1.57) Since J is a diagonal matrix, the solution of the system (A1.57) can be obtained very easily, as the vector of solutions to each variable yi, and is given by, y 1 (t) c 1 e ρ 1t (A1.58) = y n (t) c n e ρ nt Finally, we can use the transformation x = Py to return to the original variables x1,, xn, as, c 1 e ρ 1t x(t) = Py(t) = [ v 1 v n ] = c (A1.59) 1 e ρ1t v 1 + c 2 e ρ2t v 2 ++ c n e ρnt v n c n e ρ nt Thus, under the assumption that the n x n matrix A has n distinct real eigenvalues ρ1, ρn, with corresponding eigenvectors v1, vn, the general solution of the homogeneous linear system (A1.52) is given by, x(t) = c 1 e ρ1t v 1 + c 2 e ρ2t v 2 ++ c n e ρnt v n (A1.60) 15
The solution in the cases of complex eigenvalues or multiple eigenvalues without enough eigenvectors are analogous to the solution of the second order homogeneous system analyzed in section A1.4. Steady states and stability conditions are defined in an analogous way to first and second order differential equations. Assuming that the vector of g s consists of constants, one gets the nonhomogeneous system, x 1(t) a 11 a 1n x 1 (t) g 1 = " (A1.61) x + n(t) a n1 # a nn x n (t) g n The steady state, if it exists, can be derived by setting the change in the x s equal to zero. A steady state exists if A is non singular, and is given by, 1 x 1 * a 11 a 1n g 1 (A1.61) = " x n * a n1 # a nn g n where, xi* denotes the steady state value of xi. The xi* s can be regarded as equilibrium points. If the xi s are predetermined variables, for the system to converge to equilibrium, all eigenvalues must be smaller than zero. In this case the equilibrium is a fixed node. When the xi s consist of p predetermined and q non predetermined variables, where p+q=n, the equilibrium, if it exists, is a saddle point. For the system to converge to equilibrium, there must be p negative eigenvalues and q positive eigenvalues. The negative eigenvalues correspond to the predetermined variables, which are solved backwards, and the positive eigenvalues correspond to the non predetermined variables which are solved forward. Thus, a system with p predetermined and q non predetermined variable has a stable equilibrium (a saddle point) if the matrix of coefficients has p negative and q positive eigenvalues. The adjustment path is unique and is called a saddle path. 16
References Boyce W.E. and DiPrima R.C. (1977), Elementary Differential Equations and Boundary Value Problems, New York, Wiley. Chiang A. (1974), Fundamental Methods of Mathematical Economics, New York, McGraw Hill. Simon C.P and Blume L. (1994), Mathematics for Economists, New York, Norton. 17