Applied Differential Equation November 3,
Contents 5 System of First Order Linear Equations 5 Introduction and Review of matrices 5 Systems of Linear Algebraic Equations, Linear Independence, Eigenvalues, Eigenvectors 5 53 Basic Theory of Systems of first order linear equations 8 54 Homogeneous linear systems with Constant Coefficients 55 Complex Eigenvalues 3 56 Repeated Eigenvalues 5 57 Nonhomogeneous Linear System 8 58 Application of system of linear first order differential equation 59 Review of System of First-Order Linear Equation 4 6 Numerical Methods 7 6 Iteration methods 8 6 First order differential equation and roots of equation 9
Chapter 5 System of First Order Linear Equations 5 Introduction and Review of matrices Brief introduction: Example : an example of system of first order linear equaiton: { x (t) x(t) + y(t) + sin(t) y (t) x(t) + y(t) + cos(t) Example : higher order differential equations y (t) y(t) Let x(t) y (t) { x (t) y y (t) x x 4 (t) y y (3) x y Let 3 (t) x 4(t) y x (t) x 3(t) x 4(t) y x (t) x (t) x 3(t) x 4 (t) y So the system of linear equations we are to solve is x x 4 x x x 3 x x 4 x 3
Example 3 a 3 y + a y + a y + a y Let x y; x x ; x 3 x ; Then the equivalent system of linear equations is x x ; x x 3 ; x 3 (a x 3 + a x + a x ); a 3 or which is equivalent to x x x 3 y y y a /a 3 a /a 3 a /a 3 a /a 3 a /a 3 a /a 3 x x x 3 y y y General Form: x x x n a n y (n) + a n y (n ) + + a y Let x y, x x y, x n x n y (n ) a /a n a /a n a /a n a n /a n x x x n a n y (n) + a n y (n ) + + a y g(t) x x x n Let x y, x x y, x n x n y (n ) a /a n a /a n a /a n a n /a n x x x n + g(t)/a n 3
General form of n first order linear equations: n >, x (t), x (t),,x n (t) x p (t)x + p (t)x + + p n (t)x n + g (t) x p (t)x + p (t)x + + p n (t)x n + g (t) Review of Matrix Theory: (a)definition x n p n (t)x + p n (t)x + + p nn (t)x n + g n (t) m n matrix a a n A (a ij ) m n (a ij ) a m a mn vectors 3 Transpose of A: a a m A T a n a mn n m Conjugate : ā ij Real part of a ij Imaginary part of a ij Adjoint of A: A ĀT (b) Particular matrices zero matrix: square matrix A (a ij ) n n Identity matrix I (it must be a square matrix) 4 n n
diagonal matrix: A diag(a, a,, a nn ) a a a n n a nn upper triangular matrix a a a n n a nn lower triangular matrix (c) Operations and matrices a a a n n a nn Equality: same size and same elements addition: same size A m n + B m n (a ij + b ij ) m n multiplication by a number: αa (αa ij ) multiplication by two matrices: AB C A (a ij ) m n, B (b jk ) n q n C (c ik ) m q, c ik a ij b jk j (ABC) (AB)C A(BC), but AB BA Examples: Ax?, x T y?, xx? 5 Systems of Linear Algebraic Equations, Linear Independence, Eigenvalues, Eigenvectors Systems of linear Algebraic Equations coefficients are constants a x + a x + + a n x n b a x + a x + + a n x n b 5
Given Ax b, if det(a) x A b Example n x 3 x 4 Augmented matrix 4 3 4 3 3 4 x 3 x 4 3 3 Linear Independence Def A set of k vectors x (),, x (k) is said to be linearly dependent if there exists a set of (complex) numbers c,, c k At least one of which is nonzero, such that c x () + + c k x (k) ( ) On the other hand, if the only set c,, c k for which (*) is satisfied is c c k, then x (),, x (k) are said to be linearly independent Example: x (), x(k), x(3) 3 c x () + c x () + c 3 x (3) c + C 3 C + 4C 3 C C 3, C 3 C 3, C C 3 For example {C, C, C 3 } { 4, 3, } So x (), x () and x (3) are linearly dependent Eigenvalues and Eigenvectors Def A is a n n matrix If there exist a value λ and a non-zero vector x such that Ax λx then λ is called an eigenvalue of A and x is an eigenvector corresponding to λ 6
if x is a eigenvector corresponding to λ, then cx is a eigenvector corresponding to cλ λ satisfies det(a λi) if A is a Hermitian matrices (a ij) a ( ji), all eigenvalues are real Example : Find eigenvalues and eigenvectors for A 3 Example : A A λi 4 4 7 λ 4 4 7 λ Eigenvalue: ( λ)( 7 λ) + 6 λ + 6λ + 9 (λ + 3) λ λ 3 Eigenvectors: Find x such that Ax λx ( 4 4 x (A λ)x (A + 3)x 4 4 x ) 4x 4x, 4x 4 x x x ( ) ( ) ( ) x x x x x x So the eigenvector is Example 3 A A λi ( 3 ) λ 4 3 λ Eigenvalue: λ + 5, λ 5 ( λ)(3 λ) 4 λ 4λ 7
Eigenvectors: Find x such that Ax λx (A λ )x (A ( + ( ) ( + 5) 5)x x 5 x ( + ( + ) 5 5)x + x x x ( ) x ( x x + ) 5 x x + 5 x So the eigenvector corresponding to λ + 5 is (A λ )x (A ( 5)x + 5 + 5 ( 5 )x + x x 5 x So the eigenvector corresponding to λ 5 is + 5 ( x x 5 ) 53 Basic Theory of Systems of first order linear equations General form of n first order linear equations x x (t) x n (t) x p (t)x + + p n (t)x n + g (t) x p (t)x + + p n (t)x n + g (t) x n p n (t)x + + p nn (t)x n + g n (t), A(t) x A(t)x + b(t) p (t) p n (t) p n (t) p nn (t) Homogeneous System: when b(t), the linear system becomes which is called a homogeneous equation x A(t)x, b(t) g (t) g n (t) 8
Superposition Principle: If the vector functions x () x (t) x n (t) and x () x (t) x n (t) are two solutions of x A(t)x, then the linear combinations C x () + C x () is also a solution for any constants C and C Example : x x 4 x () e 3t e 3t e 3t, x () e t e t x C x () + C x () C e 3t + C e t C e 3t C e t The Wronskian Consider n solutions of x A(t)x: x (t) x () Define X(t) x (t) x n (t) x n (t) x n (t) x nn (t),, x (n) e t x n (t) x nn (t) x (), x (),, x (n) The Wronskian of these n solutions is defined by W x (),, x (n) det(x(t)) If W, these n solutions are linearly dependent, that is, one of them can be represented as a linear combination of the other solutions If W, these n solutions are linearly independent, that is, none of them can be represented as a linear combination of the other solutions In this case, each solution is essential Example (continued) e 3t X(t) e 3t e t e t W x (), x () 4e t Relation between the Wronskian of linear system and that of second-order linear equation: Linear equation: y + p(t)y + q(t)y Solutions: y (t), y (t) y (t) y (t) Wronskian: W y, y det y ( t) y (t) { Transform to linear system: y + p(t)y + q(t)y x y,x y x x x q(t)x p(t)x 9
y (t) is a solution x () x () y (t) x () y (t) is a solution y (t) is asolution x () x () y (t) x () y (t) is a solution W y, y W x (), x () y (t) y det (t) y (t) y (t) Same! General Solution: Suppose that vector functions x (),, x (n) are n solutions of x A(t)x, α t β If there exists a point t α, β such that then the general solution of x A(t)x is where C C C n Example x x () x C e t C e t C 3 e 3t W x (),, x (n) (t ) det(x(t )), x φ(t) C x () (t) + C x () (t) + + C n x (n) (t) X(t)C 3 e t, x () x general solution e t, x (3) e 3t 54 Homogeneous linear systems with Constant Coefficients Consider x Ax, A: constant matrix of n n Seek the solution of the form x ξe λt, x and ξ are vectors, λ is a single-value where λ and the constant vector ξ are to be determined x Ax λξe λt Aξe λt
Aξ λξ or (A λi)ξ This implies that if λ and ξ are an eigen-pair (eigenvalue and eigenvector) of A, then x(t) ξe λt is a solution of x Ax If A has n eigenvectors: Aξ () λ ξ (),, Aξ (n) λ n ξ (n) and W ξ (),, ξ (n), then the general solution is Examples x(t) C ξ () e λ t + + C n ξ (n) e λnt A diag(, 3, 4) Eigenvalues are, 3, 4, corresponding eigenvectors of, 3 A 3 det(a λi) λ 3 3 λ λ 3λ + 9 λ 3λ 7, λ, b ± b 4ac 3 ± 37 λ 544, λ 4544 a Since Aξ λξ (A λi)ξ ( 544) 3 x λ 544: 3 ( 544) x x x 544x + 3x x 544 x x 847 3 So ξ 847 4544 3 λ 4544: 3 4544 3 ξ 3544 x C 847 e 544t + C 3 3544 3544 3 3 544 e 4544t
A 4 det(a λi) λ 4 λ λ λ 3 (λ 3)(λ + ) λ 3, λ A λ I ξ 4 A λ I ξ 4 x C e 3t + C e t Summary: x Ax case : A Hermitian: a ij a ji Here a ji denotes the conjugate of a ji : a + bi a bi Every eigenvalues of a Hermitian matrix are real, and all the eigenvectors are linear independent case : A real but non-hermitian (A A) Aξ (i) r i ξ (i), i,, n, W ξ (i),, ξ (n) (i) All eigenvalues are real and distinct x () ξ () e r t,, x (n) ξ (n) e rnt x C ξ () e r t + + C n ξ (n) e rnt Aξ (i) r i ξ (i), i,, n x C ξ () e r t + + C n ξ (n) e rnt (ii) Some eigenvalues occur in complex conjugate pairs (a+bi, a bi), others are real and distinct (iii) some eigenvalues are repeated; others are real and distinct Example A 5 λ 3 3 5 λ 5 3 (Hermitian) 3 5 λ λ + 6 (λ 8)(λ ) λ 8 λ 5 8 3 A λ I ξ () 3 5 8
5 3 A λ I 3 5 Example A + i i ξ () (Hermitian) λ + i i λ ( λ)( λ) (+i)( i) λ 3λ+ ( i ) λ 3λ+ λ(λ 3) λ and λ 3 + i A λ I A ξ () + i i + i A λ I ξ () + i i + i x C e t + i + C e 3t + i + i C + C e 3t 55 Complex Eigenvalues To solve x Ax ( ) A is a real matrix and has a complex eigenvalue r λ + iµ: Aξ () r ξ () Aξ () r ξ () A ξ () r ξ () r is also an eigenvalue and the corresponding eigenvector is ξ () Complex eigenvalues and eigenvectors appear in pairs If (r, ξ) is an eigen-pair, and r complex, then ( r, ξ) is also an eigen-pair x () ξ () e r t, x () ξ () e r t x and ξ () are vectors, r is single-value C x () + C x () C ξ () e rt + C ξ() e r t C ξ () e rt + C ξ () e r t Let ξ () a + ib, r λ + iµ, e rt e λt cos µt + i sin µt, where a and b are real vectors ξ () e rt (a + ib)e λt (cos µt + i sin µt) e λt a cos µt + i(b cos µt + a sin µt) b sin µt ξ () a ib, r λ iµ, e r t e λt cos µt i sin µt ξ () e r t (a ib)e λt (cos µt i sin µt) e λt a cos µt i(b cos µt + a sin µt) b sin µt if C C : C x () + C x () Reξ () e r t e λt a cos µt b sin µt a real solution of ( ) if C i, C i : C x () + C x () Imξ () e r t e λt a sin µt + b cos µt another real solution of ( ) 3
Summary r λ + iµ, r λ iµ, ξ () a + bi ξ() a bi Two real solutions are e λt a cos µt b sin µt, e λt a sin µt + b cos µt If r 3,,r n real and distinct, the general solution x C e λt a cos µt b sin µt + C e λt a sin µt + b cos µt + C 3 ξ (3) e r 3t + + C n ξ (n) e rnt Example : x x A ri ( r) + r ±i r + i, r i r i A r I i ξ () +i i }{{}}{{} x C e t ( Example : x a cos t 4 b sin t) + C e t ( sin t + A ri ( r) + 4 ( r) 4 r ±i r + i, r i λ µ i 4 A r I ξ () i i ξ () + i x C e t ( cos t sin t) + C e t ( sin t + cos t) cos t) 4
56 Repeated Eigenvalues x Ax If r r is a k-fold root (r r r k ) of the equation det(a ri) then r is said to be an eigenvalue of multiplicity k of A Now we assume that r r is an eigenvalue of multiplicity Suppose Aξ () r ξ () (ξ () ) x () ξ () e rt is a solution of x Ax Seek another solution in the form: x () ξ () te r t However, by dx() dt Ax () ξ () e r t + ξ () rte r t Aξ () te r t or ξ () e r t + ξ () rte r t r ξ () te r t ξ () e r t ξ () (conflict) Seek another solution in the form: x () ξ () te r t + ηe r t By dx() dt Ax (), ξ () e r t + ξ () tr e r t + ηr e r t r ξ () te r t + Aηe r t (A r I)η ξ () ( ) since det(a r I) (*) may not have solution General solution: x C ξ () e r t + C t ξ () e r t + ηe r t where (A r I)η ξ () 5
Example x 4 8 4 x, x() A ri (4 r)( 4 r) + 6 r 6 + 6 r r r 4 A r I ξ () 8 4 (A I)η ξ () 4 η 8 4 η { 4η η 8η 4η 4η η η η + 4 η /4 + /η /4 η η / + η For any value of η, (i) is solution for η We just pick a particular one Let η, η (i) /4 x () ξ () e t ξ () x () ξ () te t + ηe t ξ () t + η t ( ) /4 x C + C t + By x() /4 C + C { C + /4C C C, C 4 4 x(t) + t 8 /4 + 6
Example : x 3 x A ri ( r)(3 r) + r 4r + 4 (r ) r r A I ξ () { (A I)η ξ () η η η + η η + η, η η η + η η Pick one particular η, η ( ) x C e t + C te t + e t In some cases, there will be two or more linear independent eigenvectors corresponding to repeated eigenvalues For example, if ɛ and ɛ are two linear independent eigenvectors to the repeated eigenvalue r, then we have two linear independent solutions ɛ e rt and ɛ e rt can be used to form the fundamental set of solution Example Find the general solution to x x To find the eigenvector for - A ri, (r + ) (r ), r, r r 3 x x x 3 x + x + x 3, x x x 3 x (x + x 3 ) x x 3 x x 3 x + x 3 Any value of x and x 3 above will yield an eigenvector 7
and linear independent solutions one particular eigenvector to r is are both eigenvectors to -, and they are linearly independent So we have two e t and e t, so another particular solution is e t These three particular solution are linear independent, so they form a fundamental set of solutions, and the general solution is x(t) C e t + C e t + C 3 e t 57 Nonhomogeneous Linear System Nonhomogeneous Linear System: x Ax + g(t) ( ) Example Find the solution to x e t x + 3t Ax + g(t) Method of Undetermined coefficients: Suppose u(t) is a general solution to the homogeneous system x Ax, then the general solution to the nonhomogeneous system is x(t) u(t) + Y(t), where Y(t) is a particular solution to (*) Soln: General solution to is x C x x e 3t + C e t 8
Write g(t) e t + 3 t Since e t is in the general solution to the homogeneous equation, we assume the particular solution takes the form Y (at + b)e t + ct + d, a, b, c and d TBD And Y should satisfy (*),that is Y AY + e t + 3 t ae t (at + b)e t + c Aate t + Abe t + Act + Ad + e t + 3 By collecting terms, we obtain the following algebraic equations for a, b, c and d Aa a a is the eigenvector to r Ab a b Ac 3 Ad c t Variation of Parameters Fundamental Matrix: Suppose that x () (t),, x (n) (t) form a fundamental set of solutions for the equation x P(t)x ( ) Then Φ(t) x () (t) x(n) (t) n (t) x (n) n (t) x () whose columns are the vectors x () (t),, x (n) (t), is said to be a fundamental matrix for the system (**) The fundamental matrix to the general solution of x Ax in (*) is given by e 3t e t Φ(t) e 3t e t 9
Then the general solution x C e 3t + C e t C Φ(t)C, C C and therefore, Φ (t)c AΦ(t)C Φ (t) AΦ(t) We assume the particular solution to (*) take the form Y(t) Φ(t)u(t), u(t) u (t) u (t) and satisfy (*), that is Since Φ (t)u(t) + Φ(t)u (t) AΦ(t)u(t) + g(t) Φ (t)u(t) AΦ(t)u(t), Φ(t)u (t) g(t) u (t) Φ (t)g(t) u(t) Φ (t)g(t)dt, pick a particular solution Laplace Transform method: We can use Laplace transform to find a particular solution, ie, the solution to x Ax + g(t), x() Take the Laplace transform at both hand sides, sy(s) AY(s) + G(s), (si A)Y(s) G(s) Y(s) (si A) G(s) Take the inverse Laplace transform Y(t) L {(si A) G(s)} Fact : The inverse of a matrix a b c d c a c d b a b d d b c a ad cb
58 Application of system of linear first order differential equation Chemical Kinetics A molecule fluctuate between two states A and B Suppose every minutes, per molecules, there are h molecules molecules changing from state A to state B, and h molecules changing from state B to A h /: rate of change from state A to state B h /: rate of change from state B to state A x (t): population of molecules at state A at time t x (t): population of molecules at state B at time t t: time in minute Model: Nested Catenary system x h x + h x x h x + h x Suppose water is flowing through 3 containers in the following graph with a constant rate r At the same time, pure water is pumping into container with the same rate r Suppose there is a certain chemical concentration in these three containers at the beginning Find the amount of this chemical concentration in these three containers at any time t r x (t) V r x (t) V r x 3 (t) r V 3
Soln x i (t): amount of chemical concentration of each container at time t v i : volume of solution of each container It is a constant value because the fluid rate of in and out is the same x i (t)/v i : density of chemical concentration rate of changerate in - rate out x x (t) r v x x (t) r x r v v x 3 x (t) r x 3 r v v 3 The system is homogeneous, because there is no external input of the chemical If the water pumped into container contains % chemical concentration, then the system is non-homogeenous or Newton Cooling Model: Newton s cooling law: x x r x (t) r v x x (t) r x r v v x 3 x (t) v r x 3 v 3 r r/v r/v r/v r/v r/v 3 x + r Temperature ratek(temperature difference)
Attic x (t) 35 Main Floor x (t) Basement x 3 (t) 45 x (t) x (t) x 3 (t) t Temperature in the attic Temperature in the main floor Temperature in the basement Time in hours Home heating: Assume it is winter time and the outside temperature in constantly 35 F during the day Also assumed is a basement earth temperature of 45 F Initially, the heat is off for several days The initial values at noon (t ) are then x 3 () 45, x () x () 35 A small electric heater is turned on at noon, with thermostat set for F When the heater is running, it provides a F rise per hour, therefore it takes some time to reach F x k (35 x ) + k (x x ) attic x k (x x ) + k (35 x ) + k 3 (x3 x) + main floor x 3 k 3 (x x 3 ) + k 4 (45 x 3 ) basement or x (k + k ) k k (k + k + k 3 ) k 3 k 3 (k3 + k 4 ) x + 35k 35k + 45k 4 3
59 Review of System of First-Order Linear Equation Section 5 Introduction x P (t)x + g(t) x Ax, A (a ij ) n n, a ij constant Relationship between higher order differential equation and system of linear differential equations General Form: x x x n a n y (n) + a n y (n ) + + a y Let x y, x x y, x n x n y (n ) a /a n a /a n a /a n a n /a n x x x n a n y (n) + a n y (n ) + + a y g(t) x x x n Let x y, x x y, x n x n y (n ) a /a n a /a n a /a n a n /a n x x x n + g(t)/a n Section 54 Homogeneous system x P (t)x (i): Superposition Theory: If x () (t) and x () (t) are solutions to (i), then C x () (t) + C x () (t) is also a solution The Wronskian W det(x (),, x (n) ) If W then x (),, x (n) are linear independent, and therefore forming a fundamental set of solutions 4
General solutions If x (),, x (n) are linear independent, then the general solution to (i) is C x () (t) + C x () (t) + + C n x (n) (t) Section 55 Homogeneous system with constant coefficients x Ax: case : A Hermitian: a ij a ji Every eigenvalues or a Hermitian matrix are real, and all the eigenvectors are linear independent Aξ (i) r i ξ (i), i,, n, W ξ (i),, ξ (n) x () ξ () e rt,, x (n) ξ (n) e rnt x C ξ () e rt + + C n ξ (n) e rnt case : A non-hermitian, that is A A, A real (i) All eigenvalues are real and distinct Aξ (i) r i ξ (i), i,, n, W ξ (),, ξ (n) x(t) C ξ () e rt + + C n ξ (n) e rnt (ii) Complex conjugate pair eigenvalues: r λ + µi, r λ µi ξ () a + ib, x () (t) e λt (a cos µt b sin µt), ξ () a ib x () (t) e λt (a sin µt + b cos µt) (iii) Repeated eigenvalues: r r real, with only one linear independent eigenvector x () (t) ξ () e r t, x () (t) ξ () te r t + ηe r t (A r I)η ξ (), η may not exist Section 57 Solutions of Non-homogeneous solution x Ax + g(t) (ii) Suppose x (t) is the general solution to the homogeneous equation x Ax (iii) and Y (t) is a particular solution to (ii), then the general solution to (ii) is x(t) x (t) + Y (t) Methods to find Y (t): 5
Method of undetermined coefficients Method of variation of parameters Fundamental matrix: suppose x () (t),, x (n) (t) are n linear independent solutions to the homogeneous equation x Ax Then the fundamental matrix The general solution to (iii) can be written as Φ(t) x () (t),, x (n) (t) x(t) Φ(t)C, C The particular solution Y (t) can be calculated as follows: u (t) Φ (t)g(t) u(t) C C n Φ (t)g(t)dt Method of Laplace transform Y (t) Φ(t)u(t) 6
Chapter 6 Numerical Methods Consider IVP Numerical methods dy dt f(t, y), y(t ) y Given a set of discrete time instants want to find approximations of the solution, ie, t < t < < t n < < t N y y(t ) y y(t ) y n y(t n ) y N y(t N ) Often: for a positive time step h > t, t t + h,, t n+ t n + h, t n t + nh, n,,, N 7
6 Iteration methods Derivation: Given y n, we can approximate y n+ according to dy dt f(t, y), y n+ y n h Key: Given y n, how to choose the best slope f(t, y) Methods of choosing slope f(x, y) The Euler or tangent line method Improved Euler Method y n+ y n h f(t, y), h t n+ t n f(t n, y n ) y n+ y n + hf(t n, y n ) y n+ y n f(t n, y n ) + f(t n+, y n+ ) h Idea: y n+ y n average of f(t, y) over y n,y n+ h f(t n, y n ) + f(t n+, y n+ ) can be interpreted as an average slope However y n+ is unkown, whereas it is to be calculated So we use the Euler method to estimate y n+ first The Runge-Kutta method ỹ n+ y + hf(t n, y n ) ( ) f(tn, y n ) + f(t n+, ỹ n+ ) y n+ y n + h y n+ y n h k n + k n + k n3 + k n4 6 where k n f(t n, y n ), k n f(t n + h, y n + hk n), k n3 f(t n + h, y n + hk n), k n4 f(t n + h, y n + hk n3 ) 8
The sum k n + k n + k n3 + k n4 can be interpreted as an average of f(t, y) 6 Comparison of three methods 3 Euler method (t, y ) Improved Euler Method exact solution y φ(t) Runge-Kutta Method t t 6 First order differential equation and roots of equation Consider the first order differential equation dy dt (y )(y + ) the qualitative analysis gives the convergence region of the initial value 9
3 4 3 3 4 5 y (x + )(x ) 3 4 If y(t ) <, then lim t y(t) If y(t ) >, then lim t y(t) However, for the inverse problem lim y(t) when y(t ) > t dy dt (y + )(y ) Conclusion: We can use the first order differential equation to find roots of equation If using the Euler method to do the discretization when h is small enough y n+ y n h (y n + )(y n ) if < y <, lim n y n if y >, then the iteration yield lim n y n y n+ y n h (y n + )(y n ) Besides, we can also use the improved Euler method and Runge-Kutta method to do the discretization Remark: To get a convergent numerical iteration, we need to choose a suitable initial value y The thumb of rule is, f (y ) should not be too large When f (y ) is very large, the iteration might diverge 3