2 Determinants The Determinant of a Matrix Properties of Determinants Cramer s Rule Vector Spaces 17

Transcription

1 Contents 1 Matrices and Systems of Equations 2 11 Systems of Linear Equations 2 12 Row Echelon Form 3 13 Matrix Algebra 5 14 Elementary Matrices 8 15 Partitioned Matrices 10 2 Determinants The Determinant of a Matrix Properties of Determinants Cramer s Rule 15 3 Vector Spaces Definition and Examples Subspaces Linear Independence Basis and Dimension Change of Basis (coordinate vector, transition matrix) Row Space and Column Space 25 4 Linear Transformations Definition and Examples Matrix Representations of Linear Transformations Similarity 30 5 Orthogonality The Scalar Product in R n Orthogonal Subspaces Least Squares Problems Inner Product Spaces Orthonormal Sets The Gram-Schmidt Orthogonalization Process 40 6 Eigenvalues Eigenvalues and Eigenvectors Systems of Linear Differential Equations Diagonalization 46 1

2 Linear Algebra with Applications (Steven Leon) August 26, Matrices and Systems of Equations 11 Systems of Linear Equations A linear equation in n unknowns (example) A linear system of m equations in n unknowns = m n system (example) 2 2 systems (show by examples and graphs) Equivalent systems def { x 1 + x 2 = 2 ex (Fig 111(i)) and x 1 x 2 = 2 { x 1 + x 2 = 2 ex (Fig 111(i)) and x 1 x 2 = 2 { 2x 1 x 2 = 4 x 1 + 2x 2 = 2 { x 1 + x 2 = 2 x 1 x 2 = 2 (Fig 111(iii)) are NOT Triangular form for n n systems 1 All entries below the diagonal are zero 2 The diagonal entries are nonzero Example: [ , { 2x1 x 2 = 5 3x 2 = 3, x 1 2x 2 +3x 3 = 6 2x 2 x 3 = 1 x 3 = 3 Triangular forms can be solved by back substitutions Question: How to solve the general linear system? Coefficient matrix and augmented matrix for a system 2

3 Elementary Row Operations: 1 Interchange two rows 2 Multiply a row by a nonzero real number 3 Replace a row by its sum with a multiple of another row Elementary row operations change a system to an equivalent system We can use elementary row operations to change a system to triangular form Examples: Ex Example 4, p9 { 2x1 +x 2 = 1 x 1 +2x 2 = 1 x 1 + 2x 2 + 3x 3 = 3 3x 1 + 2x 2 + x 3 = 1 2x 1 + x 2 + x 3 = 2 From now on, we use augmented matrices to represent the systems Homework 1c, 3, 6a,d, 10, Row Echelon Form Emphasis more on introducing the terminologies, and the process to get the reduced row echelon form Row echelon form, lead variables, free variables, reduced row echelon form Row echelon form: A matrix is in row echelon form if 1 The first nonzero entry in each row is 1 2 If row k has nonzero entries, then row k + 1 has more leading zeros than row k 3 If all entries of one row are zeros, then all entries below this row are zeros Example: Ex 2, Ex 3, p15 Indicate lead variables, free variables We can determined whether the system is consistent or inconsistent by row echelon form Reduced row echelon form is a special row echelon form, where each lead variable is the only non-zero entry in its column Ex those above Example 6, p18 The process of using elementary row operations to change a matrix into row echelon form / reduced row echelon form is called Gaussian Elimination 3

4 Example: Change the matrices into reduced row echelon forms , Process of solving a linear system: Use row operations I, II, and III to perform Linear System Augmented matrices Example 5, p17 Gaussian Elimination move free vars to the right Reduced Row Echelon Form Lead Vars, Free Vars, Consistency Solution Set Overdetermined System: more equations than unknowns (m > n) Usually (not always) inconsistent Examples: Underdetermined System: fewer equations than unknowns (m < n) Usually (not always) consistent Examples: Homogeneous System: Zeros on the right Homogeneous systems are always consistent Homework 1-4,5a,c,e,g,i,k,7,8,9,10 4

5 13 Matrix Algebra matrix, scalar, 1 A = (a ij ) = [ a 1, a 2,, a n a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn = a(1, :) a(2, :) a(m, :) row vector: a(i, :) = [a i1, a i2,, a in, i = 1,, m a 1j a 2j column vector (vector): a j = a(:, j) =, j = 1,, n a mj 2 Euclidean space R n : the set of all n 1 matrices of real numbers If A and B are m n matrices, then A = B if and only if a ij = b ij for each i and j Scalar Multiplication of a real number and a matrix Example: [ A = Matrix addition for two m n matrices Example: [ A = [, B = , 3A (Important) Matrix Multiplication and Linear Systems, A + B, A + 2B 1 Motivation: try to write a linear system as the form ax = b 2 One equation in n unknowns x 1 + 2x 2 = 3, Let A = [ 1 2, x = [ x1 2x 1 + x 2 x 3 = 5, Let A = [ 2 1 1, x = x 2, and define Ax by x 1 x 2 x 3, define Ax by 5

6 3 Generic Matrix Multiplication: m n n k = m k Restriction for AB to be meaningful: Number of the columns of A = Number of the rows of B { }} { a 11 a 12 a 1n b 11 b 12 b 1k a 21 a 22 a 2n A =, B = b 21 b 22 b 2k a m1 a m2 a mn b n1 b n2 b nk, Rule of multiplication: If [ [ a ij bij m n n k = [ c ij, then m k c ij = the i-th row of A multiples the j-th column of B c 11 c 12 c 1k c 21 c 22 c 2k AB = C =, c ij = a(i, :)b(:, j) c m1 c m2 c mk Example: [ 0 1 A = 1 2 Try: A = Try: A = [ 1 3, B = 1 0 [ [ , B =, B = 4 System of m equations in n unknowns a 11 a 12 a 1n a 21 a 22 a 2n A = = a m1 a m2 a mn, can we do AB? BA? Is AB = BA? [ a(1, :) a(2, :) a(m, :), can we do AB? BA?, can we do AB? BA?, x = x 1 x 2 x n, b = b 1 b 2 b m, Then Ax = a 11 x 1 + a 12 x a 1n x n a 21 x 1 + a 22 x a 2n x n a m1 x 1 + a m2 x a mn x n = a(1, :)x a(2, :)x a(m, :)x = b 6

7 Example: { 2x1 x 2 = 0 x 1 + x 2 = 3, x 1 2x 2 +3x 3 = 6 2x 2 x 3 = 1 x 3 = 3 (n n matrices) The identity matrix I: AI = IA = A for all n n matrices A show why exponential notation A k = AAA }{{} k times (n n matrices) The multiplicative inverse matrix A 1 : If there is B, such that AB = BA = I, we say that A is non-singular, and B = A 1 Otherwise, we say that A is singular Example: A = [ [ 2 1, B = A = [ , A singular A non-singular (m n matrices) Transpose: a 11 a 12 a 1n a 21 a 22 a 2n A =, AT = a 11 a 21 a n1 a 12 a 22 a n2 a m1 a m2 a mn a 1n a 2n a mn Example: A = [ [ 1 3, B = 1 0 Notational Rules (w/o parentheses): Operations (transpose, inverse, etc) > Multiplication > Summation Example: AB 2C T D, A 1 (2B + C T ) Algebraic Rules: Difference: AB BA 7

8 (Skip, see by yourself) Most of the others are like usual arithmetic operations: (A, B, C matrices, α, β real numbers) For transpose: Homework 1-8, Elementary Matrices Equivalent Systems A + B = B + A (A + B) + C = A + (B + C) (AB)C = A(BC) A(B + C) = AB + AC (A + B)C = AC + BC (αβ)a = α(βa) α(ab) = (αa)b = A(αB) (α + β)a = αa + βa α(a + B) = αa + αb (A T ) T = A (αa) T = αa T (A + B) T = A T + B T (AB) T = B T A T 1 Ax = b is a m n system, M is a nonsingular m m matrix, then Ax = b is equivalent to MAx = Mb (explain) 2 We can apply a series of nonsingular matrices E 1,, E k to both side of Ax = b to obtain a simpler system: E k E 1 Ax = E k E 1 b Elementary row operations = Multiple an elementary matrix from the left on both sides of Ax = b Such elementary matrix is obtained by doing the corresponding elementary row operations on I [ We use the system to illustrate the relations between elementary 1/2 3/2 1 row operations and elementary matrix multiplication 8

9 1 (Type I) Interchange two rows of I: Acting on an 3 3 matrix Example: , (Type II) Multiply a row of I by a nonzero constant Example: , (Type III) Add a multiple of one row of I to another row Example: , Elementary matrices can generate all nonsingular matrices equivalent to A iff: B = E k E 1 A, E 1,, E k are elementary matrices One way to computer A 1 for n n matrix A: Find the reduced row echelon form of [A I [A I [I B means that: So matrix B is row 1 B [A I = [BA B = [I B, so BA = I, and B = A 1 2 B is the solution of the system [A I, so AB = I Example: Ex 4, 5, p (skip) Definitions of upper triangular matrices, lower triangular matrices, diagonal matrices , 2 4 0,

10 Homework 1-6a,9a,c,e,g,12 15 Partitioned Matrices 1 We can partition a matrix by drawing horizontal lines b/t rows, and vertical lines b/t columns 2 The way we do matrix multiplication: a(1, :) A B = [ b 1 b k m n n k a(m, :) = [ Ab 1 Ab k = 3 General Situations: A 11 A 1t A = A s1 A st a(1, :)B a(m, :)B, B = C 11 C 1r AB = C =, C ij = C s1 C sr B 11 B 1r B t1 B tr, t A ik B kj Caution: We shall partition A and B in the meaningful way for multiplication k=1 # of columns of A ik = # of rows of B kj Ex A 1 [A I = [I A 1, [ A I [ I A 1 = A 1 Ex A = , B = , Try different partitions (a) normal 10

11 (b) compute, (c) (brief) 4 Definition of inner product (scalar product) and outer product Show one example Homework 1a,e,4a,c,6,8,9,13,15,18 11

12 2 Determinants 21 The Determinant of a Matrix Recall: A R n n is nonsingular (invertible), iff AB = I n for some B R n n, iff BA = I n, iff A is row equivalent to I n Determinant can be used to check whether A R n n is nonsingular The meaning and definition matrices: [a is non-singular a 0 det[a := a matrices: [ a11 a 12 a 21 a 22 if a 11 0 [ a11 a 12 a 11 a 21 a 11 a 22 row2 a 21 row1 [ a11 a 12 0 a 11 a 22 a 21 a 12 So A is non-singular a 11 a 22 a 21 a 12 0 a 11 a 12 a 21 a 22 = a 11a 22 a 21 a 12 Ex 2 2 matrices 3 n n: Ex We use the computation of Define det(a) for A = (a ij ) R n n as follow: (1) The minor of a ij : to illustrate the process M ij = the (n 1) (n 1) matrix obtained by deleting the row and column of A containing a ij (2) The cofactor of a ij : A ij = ( 1) i+j det(m ij ) (3) The determinant of A is defined by inductive way: { a 11, if n = 1 det(a) = a 11 A 11 + a 12 A a 1n A 1n, if n > 1 For each a 1j in the first row of A, we compute its minor, then its cofactor Finally we get det(a) 12

13 Thm 21 We may apply the cofactor expansion on any row or column Ex Compute by cofactor expansion along the 2nd column Ex Thm 22 det(a T ) = det(a) Thm 23 The determinant of a triangular matrix equals to the product of the diagonal elements Ex Thm 24 Let A R n n : 1 If A has a row or column consisting entirely of zeros, then det(a) = 0 2 If A has two identical rows or two identical columns, then det(a) = 0 Homework Sect 21, 1, 3cdg, 4, Properties of Determinants In this section, we study some properties of determinants, and use row operations to find a very easy way to compute determinants Thm 25 If A, B R n n, then det(ab) = det(a) det(b) We verify this by elementary matrix multiplications Elementary Row Operations: (2 2 as examples) 1 Type I Exchange two rows of A: [ [ 0 1 a11 a 12, det(ea) = det(a) = det(e) det(a) 1 0 a 21 a 22 13

14 2 Type II Multiply a row of A by a nonzero scalar: [ [ 1 0 a11 a 12, det(ea) = α det(a) = det(e) det(a) 0 α a 21 a 22 3 Type III A multiple of one row is added to another row: [ [ 1 0 a11 a 12, det(ea) = det(a) = det(e) det(a) c 1 a 21 a 22 Summary: If E is an elementary matrix: 1 Type I det(e) = α 0 Type II 1 Type III About the determinants: 1 Interchanging two rows (or columns) of a matrix changes the sign of the determinant 2 Multiplying a single row or column of a matrix by a scalar has the effect of multiplying the value of the determinant by that scalar 3 Adding a multiple of one row (or column) to another does not change the value of the determinant We can use row operations to compute determinants easily: det(ea) = det(e) det(a) = det(a) = Similar for column operations 1 det(e) det(a) Ex If one row (or column) of A R n n is a multiple of another, then det(a) = Ex (skip) Ex

15 Ex Thm 26 An n n matrix A is singular if and only if det(a) = 0 Ex Hw 16 (p105 in 7th ed) Ex (If time allows) Hw 14, 15, (p105 in 7th ed) Homework 22, 1,3f,4,6,7,12 (7th ed) 1,3f,4,6,7,10 (6th ed) 23 Cramer s Rule Lem 27 Let A R n n with cofactor A jk of a jk, then { det(a), if i = j a i1 A j1 + a i2 A j2 + + a in A jn = 0, if i j Proof: Replace the j-th row of A by its i-th row, then use cofactor expansion (along the j-th row) to compute the determinant of the new matrix Replace each term of A by its cofactor and then transpose the resulting matrix We get the adjoint of A A 11 A 21 A n1 A 12 A 22 A n2 adja = A 1n A 2n A nn One has A(adjA) = det(a)i So Ex 2 2 matrices A 1 = 1 det(a) adja Ex A =

16 Ex An integer matrix A has an integer matrix inverse iff det(a) = ±1 Thm 28 (Cramer s Rule) Let A R n n nonsingular, b R n The solution of Ax = b is (x 1,, x n ) T where: x i = det(a i) det(a) for i = 1, 2,, n, and A i is obtained by replacing the i-th column of A by b Ex x 1 + 3x 2 + x 3 = 1 2x 1 + x 2 + x 3 = 5 2x 1 + 2x 2 x 3 = 8 Homework 1c, 2ace Read Application 1 16

17 3 Vector Spaces 31 Definition and Examples Vectors and operations in R n (using graphic examples in R 2 ): 1 R n is a set of vectors: (a) Two factors determine a vector: direction and length (b) Length of a vector v = (x 1, x 2,, x n ) T of R n : v := x x x 2 n 2 Operations in R n : the sums and scalar multiplications of vectors (Show sum, scalar product, and minus graphically in R 2 ) R m n : The set of all m n real matrices vectors, sum, scalar multiplication C[a, b: The space of all continuous function on [a, b: vectors, sum, scalar multiplication P n : The set of all polynomials of degree less than n More examples of geometric objects in vector spaces (some real life examples) Vector Space Axioms: V : a set on which addition and multiplication are defined x, y, z : any elements of V, called vectors α, β : any real numbers V is a vector space, iff it is closed under addition and multiplication, ie x + y V, αx V, plus the addition and multiplication are similar to those in R n Precisely, they satisfy the axioms for any x, y, z V and α, β R: A1 x + y = y + x A2 (x + y) + z = x + (y + z) A3 there is 0 V, such that x + 0 = x A4 there is x V, such that x + ( x) = 0 A5 α(x + y) = αx + αy A6 (α + β)x = αx + βx A7 (αβ)x = α(βx) A8 1 x = x 17

18 Ex Check for P n Ex Similar for C[a, b (skip) Additional properties of vector spaces: 1 0x = 0 2 x + y = 0 y = x 3 ( 1)x = x They can be deduced from the axioms (skip) Ex Hw 16 (7th ed) Homework Sect 31, 1, 4, 7, 8, Subspaces Review: A vector space is a set V with + and such that V is closed under + and, and satisfies 8 axioms We now describe the vector spaces inside a vector space V {[ } 0 Ex S = x 2 R Then S is a nonempty subset of R 2 (Verify that S is closed x 2 under + and Demonstrate graphically) 1 Def If S is a nonempty subset of a vector space V, and S is closed under additions and scalar multiplications, that is, (a) x + y S for any x S and y S, (b) αx S for any α R and x S, then S is called a subspace of V A subspace is always a vector space In a vector space V, both {0} and V are subspace of V The other subspaces are called proper subspaces To prove or disprove whether a subset S of a vector space V is a subspace, we should check whether S is closed under additions and scalar multiplications Ex Example 2 (p125 in 7th ed) Ex Example 3 (p125 in 7th ed, show by graphic demonstration) 18

19 Ex (skip) (a) S = {(x 1, x 2, x 3 ) T x 1 + 2x 3 = 0} is a subspace of R 3 (b) S = {(x 1, x 2, x 3 ) T x 1 + 2x 3 = 1} is not a subspace of R 3 Ex S = {A R m n the entries of row two of A are all 0 s} is a subspace of R m n Ex (skip) V = P 4, S = {f(x) P 4 f (2) = 0} 2 Nullspace of a Matrix Let A be a m n matrix The nullspace of A is: Check: N(A) is a subspace of R n N(A) = {x R n Ax = 0} To find the nullspace of a matrix A, we should solve Ax = 0 Ex (skip) The nullspace of Ex Find the nullspace of 3 Span and Spanning Sets A = A = (a) Assumption and Definition [ V : vector space v 1, v 2,, v n : vectors in V a linear combination of v 1,, v n : α 1 v α n v n, for α 1,, α n R Span(v 1,, v n ) : the set of all linear combinations of v 1,, v n Ex The nullspace in the previous example Ex If v 1,, v n are vectors in a vector space V, then Span(v 1,, v n ) is always a subspace of V (Show by Span(e 1, e 3 ) in R 3 ) (b) Def {v 1,, v n } is a spanning set of V, iff every vector in V is a linear combination of v 1,, v n Ex Ex 11, p140 Ex (skip) Show that (2, 0, 0), (3, 1, 0), ( 1, 2, 1) is a spanning set of R 3 Ex (skip) Show that (1, 2, 3), (4, 5, 6), (7, 8, 9) is NOT a spanning set of R 3 Ex Show that 2x + 1, x 2 1, (x + 1) 2 is a spanning set of P 3 19

20 Homework Sect 32, 1abc, 2, 4ad, 9ad, 10ad 33 Linear Independence Motivation: We hope to find a minimal spanning set of a vector space V {[ [ [ } Ex,, spans R Question: Suppose V = span(v 1,, v n ) Is there any redundancy in v 1,, v n? Ex If v 2,, v n span V = span(v 1,, v n ), then v 1 as an element of V can be written as a linear combination of v 2,, v n That is, v 1 = a 2 v 2 + a n v n for a 2,, a n R Equivalently, v 1 a 2 v 2 a n v n = 0 So the linear system has nonzero solution (1, a 2,, a n ) 1 Def Let v 1,, v n V If c 1 v 1 + c 2 v 2 + c n v n = 0 c 1 v 1 + c 2 v 2 + c n v n = 0 has only one solution: c 1 = c 2 = = c n = 0, then v 1,, v n are linearly independent Otherwise, v 1,, v n are linearly dependent Ex If v 1 + v 2 v 3 = 0, then v 3 = v 1 + v 2 So αv 1 + βv 2 + γv 3 = Span(v 1, v 2, v 3 ) = Span(v 1, v 2 ) Another way: If v 1 + v 2 v 3 = 0, then v 1 = v 2 + v 3 So Conclusion: αv 1 + βv 2 + γv 3 = Span(v 1, v 2, v 3 ) = Span(v 2, v 3 ) v 1,, v n are linearly independent c 1 v 1 + c 2 v 2 + c n v n = 0 has only one solution c 1 = = c n = 0 The system c 1 v 1 + c 2 v 2 + c n v n = 0 has no free variable 20

21 v 1,, v n are linearly dependent c 1 v 1 + c 2 v 2 + c n v n = 0 has nonzero solution c 1 v 1 + c 2 v 2 + c n v n = 0 has free variable(s) Some v i is a linear combination of the other n 1 vectors These n 1 vectors span the space Span(v 1,, v n ) Def A minimal spanning set of V is called a basis of V 2 Geometric interpretations (Use R 2 and R 3 to demonstrate): p 148 in 6th ed, p137 in 7th ed 3 How to check linear independence/dependence? Solve the linear system c 1 v c n v n = 0 If the system { has one or more free variable(s) = linear dependent Ex Ex 5 in p 140 (7th ed) has no free variable = linear independent Thm 31 Let v 1, v n be n vectors in R n, and the matrix X = [v 1,, v n Then v 1,, v n are linearly dependent, iff X is singular, ie det(x) = 0 Ex Ex (skip) , 1, 0, , 5, , 3, 0, Ex [ 1 2, 3 4 Ex In P 4, consider x 3 x, 2x 2 + 3, 4x 3 + 3, 21 [

22 Homework Sect 33 1ce, 2ce, 3ce, 4a, 6c, Basis and Dimension A minimal spanning set {v 1,, v n } of V is a basis of V Minimal can be replaced by linear independent 1 Def {v 1,, v n } is a basis of vector space V, iff (a) v 1,, v n are linearly independent (b) v 1,, v n span V Ex Ex 1(p146 in 7th ed) In general, {v 1,, v n } is a basis of R m, if and only if n = m and det[v 1,, v n 0 Ex R 3 : 1 0, 0 0 1, To check whether {v 1,, v n } is a basis of a general vector space V or not (a) For an arbitrary vector b, write down the system α 1 v α n v n = b (b) Change the system into a linear system Ax = b, where x = (c) If A is a square matrix and det A 0, then {v 1,, v n } is a basis of V Ex (skip) R 2 2 Ex 2(p146 in 7th ed) Ex P 3 : {x 2 + 1, x 2 2x 1, 3} 3 Standard basis 4 (a) R n (b) R m n (c) P n α 1 α n 22

23 Thm 32 If V = Span(v 1,, v n ), and m > n, then any m vectors of V are linearly dependent Cor 33 If {v 1,, v n } and {u 1,, u n } are both bases for V, then m = n We call such n as the dimension of V 5 Def If V has a basis consisting of n vectors, we say that V has the dimension n, written as dim V = n Define dim{0} = 0 finite-dimensional: There is a set of finite vectors spanning V Otherwise, V is infinite-dimensional Ex R n, R m n, P n, are finite dimensional Ex P, C[a, b, are infinite dimensional Ex Let A R m n Then dim N(A) = # of free variables in [A 0 6 Summary: TFAE: (a) dim V = n (b) V = Span(v 1,, v n ) for some linearly independent vectors v 1,, v n (c) Any n linearly independent vectors in V span V (d) Any n vectors spanning V are linearly independent (e) Any linearly independent vectors in V can be expanded to a basis (of n vectors) of V (f) Any vectors spanning V can be pared down to a basis (of n vectors) of V Ex HW 10 (p151 in 7th ed) Homework Sect 34 3, 5, 8, Change of Basis (coordinate vector, transition matrix) Every vector space looks like R n by using coordinate vectors wrt a basis Many applications can be simplify by switching from one coordinate system to another Let us focus on V = R n in this section 1 The situation of R 2 23

24 (a) Let E = [e 1, e 2 be the standard basis A vector x R 2 can be uniquely expressed as x = x 1 e 1 + x 2 e 2 x 1, x 2 : The coordinates of x relative to the standard basis E [ x1 x 2 : The coordinate vector of x wrt the standard basis E, [ x1 written as [x E = (b) For any ordered basis F = [v 1, v 2 of R 2, a vector x R 2 can be uniquely expressed as x = x 1v 1 + x 2v 2 x 1, x 2: The coordinates of x relative to the basis F [ x 1 x 2 : The coordinate vector of x wrt the basis F [ x written as [x F = 1 Ex Let F = [u 1, u 2, u 3 be a basis of V, and x = 3u 1 u 2 + 2u 3 What is [x F? 2 Given two bases E, F of R 2, and a vector v R 2, what s the relationship between [v E and [v F? [ [ 1 2 Ex R 2, E = {e 1, e 2 }, F = {u 1, u 2 } = {, } 2 3 x 2 x 2 v = au 1 + bu 2, [v F and [v E In R n, let E = [e 1,, e n be the standard basis, and F = [u 1,, u n be another basis Then for every v R n we have [v E = U[v F, U = [u 1,, u n Ex Application 1, Population migration (p 153 in 7th ed) (Markov process) 3 Def For two ordered bases E, F of V, and U C n n, TFAE: (a) U is the transition matrix from F to E (b) [v E = U[v F (c) [v F = U 1 [v E (d) U 1 is the transition matrix from E to F 24

25 The notations basis, coordinate vectors, transition matrix work for all vector spaces All finite dimensional vectors spaces are essentially R n by coordinate vectors Ex Ex 2 (p155 in 7th ed) 4 Let E, F, H, be bases of V If [v E = U[v F, and [v E = V [v H, then [v H = V 1 [v E = V 1 U[v F (We can compute the transition matrices between bases by matrix multiplications) Ex R 2, Find the transition [ [ matrix from [ F to H: [ E = {e 1, e 2 }, F = {, }, H = {, [ 1 }, v = 1 An easy way to find the transition matrix from basis F to basis H in R n : Let U R n n be formed by the vectors of F, and V R n n be formed by the vectors of H, then [V U row ops V 1 [V U = [I V 1 U (reduced row echelon form), where V 1 U is the transition matrix from F to H (Caution: The matrix U for basis F is put on the right See HW11, p162 in 7th ed) Ex Redo the previous example Ex Problem 3a in the Homework (p161 in 7th ed) Homework Sect 35, p 173 1b, 2b, 3b, 4, 6 36 Row Space and Column Space 1 Def Row space (resp column space): The space spanned by the rows (resp columns) of a matrix A a 11 a 1n A = a m1 a mn Ex The column space and the row space of A [ A = Row operations do NOT change the row space of a matrix 25

26 Thm 34 Two row equivalent matrices have the same row space Ex See above example Thm 35 A linear system Ax = b is consistent iff b is in the column space of A 2 Def The rank of a matrix A rank A = the dimension of the row space of A = the dimension of the column space of A = the number of leading variables in A Thm 36 A C m n, the rank of A plus the nullity of A equals n Proof: Change A to RREF rank A + dim N(A) = n Ex Show that rank A = rank A T for any matrix A Ex HW 1c (p167 in 7th ed) Ex Find the dimension of the subspace of R 3 spanned by 1 2, 3 4 5, Homework Sect 36, 6th ed: 1a, 2a, 4ab, 12 7th ed: 1a, 2a, 4ab, 15 Hint for the last problem: Theorem 362, definition of rank, Theorem

27 4 Linear Transformations The operations + and provide a linear structure on vector space V We are interested in some mappings (called linear transformations) between vector spaces L : V W, which preserves the structures of the vector spaces 41 Definition and Examples 1 Demonstrate: A mapping between two sets L : V W Def Let V and W be vector spaces A mapping L : V W is called a linear transformation iff L(αv 1 + βv 2 ) = αl(v 1 ) + βl(v 2 ), for all v 1, v 2 V, α, β R Equivalent Condition: L is a linear transformation iff L(v 1 + v 2 ) = L(v 1 ) + L(v 2 ) L(αv) = αl(v) Question: How does a linear map look like? Ex Linear transformations on R 1 (L : R 1 R m ) Ex Several examples on R 2 : (Show the graphs for a-d, check conditions for a,b) (a) L : R 2 R 2, L(v) = 2v (b) Ex 2 in textbook (p177 in 7th ed), L(x) = x 1 e 1 [ x1 (c) Ex 3 in textbook (p177 in 7th ed), L(x) = x 2 [ x2 (d) Ex 4 in textbook (p178 in 7th ed), L(x) = (e) L : R 2 R 1, Ex Identity map L ([ x1 x 2 ) = x 1 + x 2 Ex For any A R m n, define L A : R n R m by Check that L A is a linear transformation x 1 L A (v) := Av for v R n 27

28 Ex (Counterexample) L : R 2 R 2 defined by L(v) = v + e 1 Then L is NOT a linear transformation Ex (Counterexample) L : R 2 R 1 defined by L(x) = x x 2 2 Then L is NOT a linear transformation Ex Ex 9 (p180 in 7th ed), L : C[a, b R 1, defined by L(f) := b a f(x)dx Ex L : P n P n 1 defined by L(f(x)) = f (x) Linear transformations send subspaces to subspaces HW 12, p183 If L : V W is a linear transformation, then L(α 1 v 1 + α 2 v α n v n ) = α 1 L(v 1 ) + α 2 L(v 2 ) + + α n L(v n ) 2 The Image and Kernel Def Let L : V W be a linear transformation The kernel of L is ker(l) = {v V L(v) = 0 W }, (So ker(l) V ) Let S be a subspace of V The image of S is L(S) = {L(v) v S}, (So L(S) W ) L(V ): The image of V is called the range of L Ex The kernel and images of L are subspaces Ex Let A R m n Let L A : R n R m be defined by L A (x) := Ax The kernel of L A is exactly N(A) The range of L A is which is exactly the column space of A L A (R n ) = {Ax x R n }, Ex HW 17b, (p184 in 7th ed) Find the kernel and the range of linear operator L x 1 on R 3, where L(x) = x 2 0 Homework Sect 41, 6th ed 1ade, 3, 6bd, 15, 16ac 7th ed 1ade, 3, 6bd, 16, 17ac 28

29 42 Matrix Representations of Linear Transformations 1 All linear transformations from R n to R m are of the form L(x) = Ax for some A Thm 41 Given a linear map L : R n R m, there is A R m n, such that L(x) = Ax, ie L(x) = L A (x) In fact, A = [ L(e 1 ), L(e 2 ),, L(e n ) A is called the standard matrix representation of L Proof of the theorem Ex Ex 1, p 185 in 7th ed Ex Ex 2, p186 in 7th ed Ex (skip) Ex (skip) x 1 x 2 L : R 2 R 3, L(x) = x 1 + 2x 2 x 1 L : R 2 R 1, L(x) = x 1 + x 2 2 Thm 42 If E = [v 1,, v n is a basis of V, and F = [w 1,, w m is a basis of W, for each linear transformation L : V W, there is A R m n such that [L(v) F = A[v E for v V In fact, A = [ [L(v 1 ) F,, [L(v n ) F (Refer to Fig 422 in p188) Ex Example 4, p 188 in 7th ed Ex Let F = [b 1, b 2 be a basis of R 2, where [ 2 b 1 = 1 Find the matrix A representing x 1, b 2 = [ 1 1 L : R 3 R 2, L( x 2 ) = (x 1 + 2x 2 )b 1 + (3x 1 + 4x 2 )b 2 x 3 29

30 relative to the standard basis E of R 3 and the basis F of R 2 3 (Important) The above theorem need to compute coordinate vectors In practical, we use the following result: Thm 43 Let A be the matrix representing L : R n R m with respect to the bases E = [u 1,, u n and F = [b 1,, b m, Then the RREF of [b 1,, b m L(u 1 ),, L(u n ) is exactly [I A Ex Revisit the previous example Ex Ex 6 (p190 in 7th ed) Ex Application 1 (p191 in 7th ed) Computer graphics and animation Homework Sect 42 6th ed: 2, 4, 6, 16a, 7th ed: 2, 4, 6, 18a 43 Similarity 1 A square matrix B R n n is similar to A R n n, iff there is a non-singular matrix S R n n, such that B = S 1 AS (a) A is similar to A itself: A = I 1 n AI n (b) If A is similar to B, then B is similar to A: B = S 1 AS = A = SBS 1 = (S 1 ) 1 B(S 1 ) (c) If A is similar to B and B is similar to C, then A is similar to C: So similarity is an equivalent relationship 2 Similarity is important in representing a linear transformation by different bases Question: Let E be the standard basis in R n Let L : R n R n has the standard matrix representation L(x) = Ax If F is another basis of R n, what is the matrix representation B of L with respect to F (That is, [L(v) F = B[v F )? Answer: Let U be the transition matrix from F to E [L(v) E = A[v E = U[L(v) F = AU[v F = [L(v) F = U 1 AU[v F So B = U 1 AU 30

31 Thm 44 Two square matrices A and B are similar, if and only if both are representing a same linear transformation in different bases Ex Example 2 in the textbook (p204 in 7th ed) Method 1: Matrix Representation Theory Method 2: Transition matrix The importance of changing bases: to simplify linear transformations Ex problem 4 (p205 in 7th ed) Ex problem 9 (p206 in 7th ed) Homework Sect 43 1ae, 2, 7, 11, 12 31

32 5 Orthogonality 51 The Scalar Product in R n 1 Def Let x, y R n 2 (a) The scalar product of x and y is x T y = x 1 y x n y n (b) The Euclidean length of vector x is x = x x 2 n = x T x (c) The distance between x and y is x y = (x 1 y 1 ) (x n y n ) 2 They are natural generalizations of the situations in R 2 and R 3 [ 3 Ex R 2 : the scalar product, lengths and distance for x = and y = 4 Ex Given x 0, the vector [ 1 7 x is a unit vector It has the same direction as x x Thm 51 If x and y are nonzero vectors in R n and θ is the angle between them, then x T y = x y cos θ Proof The proof in the textbook (p212 in 7th ed) is also true for R n situations Remark: The angle θ between x and y can be computed by: cos θ = xt y x y Cor 52 (Cauchy-Schwarz Inequality) For x, y R n, we have Draw graphs to show them x T y x y 3 Def (Orthogonality) Two vectors x and y in R n are orthogonal iff x T y = 0 Ex The standard basis vectors in R 3 Ex Ex 4 (p213 in 7th ed) 32

33 4 (Show by graphs) Given x, y, the scalar projection of x onto y is: ( ) y α = x T = xt y y y, the vector projection of x onto y is: p = α y y = xt y y T y y Ex Ex 5, p214 in 7th ed (Fig 513) Let Q be the point on the line y = 1 x that is 3 closest to the point (1, 4) Determine the coordinates of Q Ex Ex 6, p215 in 7th ed Find the equation of the plane passing thru (2, 1, 3) 2 and normal to N = 3 4 Ex Ex 7, p215 in 7th ed Find the distance from (2, 0, 0) to the plane x + 2y + 2z = 0 5 Applications of orthogonality: Ex (skip) Application 1 (p217 in 7th ed) Information Retrieval Revisited Ex Application 2 (p219 in 7th ed) Statistics Correlation and Covariance Matrices (detail) Ex Application 3 (p222 in 7th ed) Psychology Factor Analysis and Principal Component Analysis Homework Sect 51 1ac, 2ac, 3a, 7, 9, Orthogonal Subspaces 1 Def Two subspaces X, Y R n are orthogonal to each other, ie X Y, iff x T y = 0 for x X, y Y Ex Example 2 in the textbook(p226 in 7th ed) 2 Def Let Y be a subspace of R n The set of vectors orthogonal to all vectors of Y form a subspace Y, called the orthogonal complement of Y Namely, Y = {x R n x T y = 0 for every y Y } Remark If X Y, then X Y = {0} 33

34 Ex In R 3, compute Y for Y = span(e 1 + e 2 ) 3 Def Given A R m n, denote the range of A by R(A) R(A) = {b R m b = Ax for some x R n } = L A (R n ) = the column space of A Similarly, R(A T ) = {y R n y = A T x for some x R m } = the row space of A Thm 53 (Fundamental Subspace Theorem) A R m n, then N(A) = R(A T ) and N(A T ) = R(A) It is one of the most important theorems in this chapter Proof x N(A) Ax = 0 Similarly for the other equality x is orthogonal to every row vector of A x is orthogonal to the row space of A x R(A T ) Ex HW1b Determine bases for R(A T ), N(A), R(A), N(A T ): [ A = Properties of orthogonal complements: Thm 54 Let S be a subspace of R n (a) dim S + dim S = n (b) R n = S S (That is, every vector v R n can be uniquely expressed as v = u + w for some u S and w S ) (c) (S ) = S Remarks: 34

35 For (a): If S is the row space of a matrix A R m n, then S = N(A) and dim S = rank A So dim S + dim S = rank A + dim N(A) = n For (c): Therefore, if T is the orthogonal complement of S, then S is the orthogonal complement of T We simply say that S and T are orthogonal complements Ex HW 9 (p233 in 7th ed) If A R m n has rank r, what are the dimensions of N(A) and N(A T )? (Two ways: 1 rank A + dim N(A) = n 2 Fundamental Subspaces Theorem) Homework p247 1ad, 2, 5, 6 53 Least Squares Problems 1 A linear system Ax = b may be inconsistent However, we can always find ˆx such that b Aˆx is minimal (which will be the distance from b to R(A)) Such ˆx is called the least squares solution of Ax = b (Demonstrate by graph, why call least squares?) Importance: We can find the best approximation for every linear system Thm 55 (531) Given a subspace S and a point b R n, there is a unique point p S such that b p minimal (Demonstrate by graph) 2 How to find ˆx such that b Aˆx is minimal? (graph) If b Aˆx is minimal, then (b Aˆx) R(A) That is, (b Aˆx) R(A) = N(A T ) (Fundamental Subspace Theroem) Equivalently, A T (b Aˆx) = 0 Thus A T Aˆx = A T b, ie ˆx is the solution of A T Ax = A T b Thm 56 Let A R m n The least squares solution(s) ˆx of the system Ax = b is the solution set of A T Ax = A T b (multiply both sides of the original system by A T ) Namely, A T Aˆx = A T b b Aˆx is minimal (The vector p = Aˆx is the projection of b onto R(A), and r(ˆx) := b Aˆx is the residue) Cor 57 (532) If A R m n has rank n, then A T A is nonsingular So the least squares problem Ax = b has the unique solution: A T Aˆx = A T b = ˆx = (A T A) 1 A T b 35

36 The projection vector p = Aˆx is the elements of R(A) that is closest to b Ex Application 2, p238 in 7th ed Spring constants Ex Ex 1, p238 in the 7th ed Ex local (mile) highway (mile) gas (gallon) Find the local and high way milages (Local: 21 mil/gal, Highway: 30 mil/gal) (refer to the textbook) Given a table of data x x 1 x 2 x m y y 1 y 2 y m we wish to find a linear function y = c 0 + c 1 x that best fit the data If we let then we get the system y i = c 0 + c 1 x i for i = 1,, m 1 x 1 1 x 2 1 x m [ c0 c 1 y 1 y 2 = The least squares solution of this system produces a linear function y = c 0 + c 1 x that is best fit the data Ex Ex 2, p240 in the 7th ed y m Ex (skip) Relationship between study time T and scores S: T (hours) S (points) We estimate that S = a + bt Find the best a and b values (a 218, b 68) Homework Sect 53 1c, 2c, 3, 4, 5 36

37 54 Inner Product Spaces 1 Examples: Standard inner product on R n (scalar product) (3 conditions) Standard inner product on R m n 2 Let V be a vector space An inner product, is an operation on V, such that for every x, y V, there is a real number x, y satisfying (a) x, x 0 with equality iff x = 0 (b) x, y = y, x (c) αx + βy, z = α x, z + β y, z for all x, y, z V and all α, β R A vector space with an inner product is called an inner product space Two vectors u, v in an inner product space are orthogonal, iff u, v = 0 The norm of an vector v in an inner product space is v = v, v Ex In R 3, define x, y = 3x 1 y 1 + 2x 2 y 2 + x 3 y 3 It is an inner product The norm of x = (1, 0, 0) T is x = x, x = 3 Ex Example 2 (p248 in 7th ed) Standard basis for Fourier analysis Thm 58 (The Pythagorean Law) If u and v are orthogonal in an inner product space (ie u, v = 0), then u + v 2 = u 2 + v 2 Thm 59 (The Cauchy-Schwarz Inequality) In an inner product space, u, v u v 3 (skip) A vector space V is a normed linear space iff for every v V, there is a real number v R (called the norm of v), such that: (a) v 0 with equality iff v = 0 (b) αv = α v for all α R (c) v + w v + w for all v, w V 37

38 55 Orthonormal Sets A standard basis in an inner product space V is the orthonormal basis 1 Def V is an inner product space with inner product, v 1,, v n satisfy that v i, v j = 0 whenever i j then {v 1,, v n } is an orthogonal set of vectors Def An orthonormal set of vectors is an orthogonal set of unit vectors orthonormal=orthogonal+normal Namely, {u 1,, u n } is orthonormal iff { 1, when i = j u i, u j = 0, when i j We abbreviate orthonormal as ON Ex x, y = x T y Then the followings are orthogonal sets: {e 1, e 2, e 3 }, { 1, 1, 0 }, {v 1 = 2, v 2 = 1, v 3 = 2 } However, only {e 1, e 2, e 3 } is an orthonormal set The third set {v 1, v 2, v 3 } can originate an orthonormal set { v 1, v 2, v 3 } v 1 v 2 v 3 Whenever {v 1,, v n } is an orthogonal set of nonzero vectors, we can form an orthonormal set {u 1,, u n } by u i := 1 v i v i Write U = [u 1,, u k Then {u 1,, u k } is an orthonormal set (with resp to scalar product) if and only if U T U = I k [ [ cos θ sin θ Ex Show that {, } is an orthonormal set for any θ sin θ cos θ 2 Def A basis of orthonormal set is an orthonormal basis An orthonormal basis in an inner product space is like the standard basis in R n Vectors in an orthonormal set are linearly independent It is easy to write a vector as a linear combination of orthonormal basis vectors Thm 510 (552) If {u 1,, u n } is an orthonormal basis, and v = n i=1 c iu i, then c i = v, u i 38

39 (proof) With orthonormal bases, an inner product is equivalent to the scalar product Cor 511 (553, 554) If {u 1,, u n } is an orthonormal basis, then for u = n i=1 a iu i and v = n i=1 b iu i, n n u, v = a i b i, u 2 = u, u = a 2 i (proof) 3 Orthogonal Matrices i=1 Def A matrix Q R n n is an orthogonal matrix iff the column vectors form an orthonormal set In other words, Q T Q = I n Ex [ cos θ sin θ Q = sin θ cos θ The linear transformation L(x) = Qx is the rotation of θ degree on R 2 Ex Permutation matrices are orthogonal matrices Prop 512 Let, be the scalar product on R n A matrix Q R n n is an orthogonal matrix iff any of the following is true: (a) The column vectors of Q form an orthonormal basis of R n (b) Q T Q = I n (or QQ T = I n ) (c) Q T = Q 1 (d) Qx, Qy = x, y for all x, y R n (e) Qx = x for all x R n Ex HW 19 (p272 in 7th ed) 4 Orthonormal sets and least squares problems Thm 513 If the columns of A are orthonormal, then the least squares solution of Ax = b is ˆx = A T b Thm 514 Let S be a subspace of V, and {x 1,, x m } be an orthonormal basis of S Then the projection of x V on S is (Fig 552) p = m x, x i x i i=1 Proof Verify (x p) x i for i = 1,, m Also p S = span(x 1,, x m ) i=1 39

40 Homework Sect 55 1, 2, 3, 6, 11, The Gram-Schmidt Orthogonalization Process 1 Idea: Starting from a basis [x 1,, x n of an inner product space, we construct an orthonormal basis [u 1,, u n such that span (u 1,, u k ) = span (x 1,, x k ) for k = 1,, n Methodology: If a subspace S has an orthonormal basis [u 1,, u k, and v is a vector outside S, then p = k i=1 v, u i u i is the projection of v onto S, and u k+1 := v p is a unit vector orthogonal to S So the orthonormal set [u v p 1,, u k is extended to [u 1,, u k, u k+1 (Show by figure) Thm 515 (Gram-Schmidt Process) Let [x 1,, x n be a basis of the inner product space V Let u 1 = x 1 x 1, u k+1 = x k+1 p k x k+1 p k for k = 1,, n 1, where p k = x k+1, u 1 u x k+1, u k u k is the projection of x k+1 onto span (u 1,, u k ) Then [u 1,, u n is an orthonormal basis of V and Ex Example 2 (p276 in 7th ed) Ex HW 5 (a) (p282 in 7th ed) span (u 1,, u k ) = span (x 1,, x k ) 2 Thm 516 (QR Factorization) If A R m n has rank n (so m n), then A = QR, where the columns of Q R m n are orthonormal, and R R n n is a nonsingular upper triangular matrix Proof Apply Gram-Schmidt process to the columns a 1,, a n of A We get an orthonormal set [u 1,, u n with span (u 1,, u k ) = span (a 1,, a k ) for k = 1,, n 40

41 So a k is a linear combination of u 1,, u k, say Therefore, where Q = [u 1,, u n and R = a 1 = r 11 u 1, a 2 = r 12 u 1 + r 22 u 2, a n = r 1n u r nn u n r 11 r 12 r 1n A = [ 0 r 22 r 2n u 1,, u n = QR, 0 0 r nn [ r11 r 12 r 1n 0 r 22 r 2n 0 0 r nn Ex Example 3 (p278 in 7th ed) Brief introduction only 3 (skip) Use Gram-Schmidt QR factorization to solve the least squares problem Thm 517 (563, p295) If A = QR is the QR factorization, then the least squares solution of Ax = b is given by ˆx = R 1 Q T b Ex Example 4 (cf Example 3) Ex HW 5 Homework Sect 56, 3, 8 41

42 6 Eigenvalues Eigenvalues are a common part of our life: vibrations, light transmission, tuning guitar, design buildings and bridges, washing machine, Partial differential problems, water flow, The simplest linear operator is L(x) = λx We hope to decompose a linear operator into some simple components like this 61 Eigenvalues and Eigenvectors Def A R n n, a scalar λ R is an eigenvalue of A if there is a nonzero vector x (called an eigenvector) such that Ax = λx Prop 61 Def λ is an eigenvalue of A Ax = λx for some nonzero vector x (A λi)x = 0 has nonzero solution N(A λi) {0} det(a λi) = 0 1 det(a λi) is the characteristic polynomial of A, and det(a λi) = 0 is the characteristic equation of A 2 N(A λi) is the eigenspace of A with resp to eigenvalue λ Every nonzero vector in N(A λi) is an eigenvector of A with resp to λ Process to solve the eigenvalue problem of a matrix A R n n : 1 Set up characteristic equation det(a λi) = 0 2 Solve the equation to get eigenvalues λ 1,, λ n (repetions are allowed) 3 For each eigenvalue λ i (1 i n), solve (A λ i I)x = 0 to get the nullspace N(A λ i I), which is the eigenspace of A corresponding to λ i Ex (Example 3, p303 [ in 7th ed) Find the characteristic equation, eigenvalues, and 3 2 eigenvectors of A = 3 2 Ex HW 1(k) (p310 in 7th ed) Ex HW 1(j) (p310 in 7th ed) 42

43 Ex HW 2 (p310 in 7th ed) Application 1: Structures Buckling of a Beam (p304 in 7th ed) A real matrix A may have some complex eigenvalues If A R n n, then det(a λi) is a real polynomial Thus the complex eigenvalues of A contain some real numbers as well as some conjugate pairs (ie if λ = a + ib (a, b R) is an eigenvalue of A, then so is λ = a ib) Ex The eigenvalues of a permutation matrix A = are the roots of λ = 0 That is, λ { 1, i, i } Thm 62 If A and B are similar, then A and B have the same characteristic polynomial, and hence they have the same eigenvalues Proof Suppose A = SBS 1, then det(a λi) = det[sbs 1 λss 1 = det[sbs 1 S(λI)S 1 = det[s(b λi)s 1 = det(s) det(b λi) det(s 1 ) = det(b λi) More properties: 1 The product of eigenvalues of A equals to det A 2 The sum of eigenvalues of A equals to tr(a) (the trace of A) That is, n λ i = i=1 n a ii i=1 3 Similar matrices have the same determinant and trace Homework Sect 61, 1bil, 2, 3, 4, 5, 8, 9 43

44 62 Systems of Linear Differential Equations 1 A system of linear differential equations has the form If we write A = [ a ij n n, Y = system becomes Y = AY y 1 = a 11 y 1 + a 12 y a 1n y n y 2 = a 21 y 1 + a 22 y a 2n y n y n = a n1 y 1 + a n2 y a nn y n y 1 y n, Y = y 1 y n, then the linear differential Ex (n = 1) The solution of y (t) = ay(t) (a R) is y = ce at (c R) Thm 63 Suppose A R n n has distinct eigenvalues λ 1,, λ n Let {x 1,, x n } be a basis of R n, where x i is an eigenvector of A corresponding to λ i Then the general solutions of the linear differential system Y = AY is: Y = c 1 e λ 1t x 1 + c 2 e λ 2t x c n e λnt x n, c 1,, c n R Ex Verify that Y i = e λ it x i is a solution of Y = AY for 1 i n Note: If such basis {x n,, x n } exists, let X = [x 1,, x n and D = λ 1 λ n then AX = A[x 1,, x n = [Ax 1,, Ax n = [λ 1 x 1,, λ n x n λ 1 = [x 1,, x n λ n = XD So X 1 AX = D (or equivalently A = XDX 1 ), and A is called diagonalizable Ex HW 1a (p323 in 7th ed) Def Initial value problem Y = AY, Y (0) = Y 0 When A has distinct eigenvalues, we can find a unique solution for the initial value problems (First, get the general solutions of the linear differential system Second, use the initial values and the general solutions to get the exact solution) 44

45 Ex HW 2d (p323 in 7th ed) Application 1: Mixture (p315 in 7th ed) 2 Higher order systems Given an equation y (n) + a n 1 y (n 1) + + a 1 y + a 0 y = 0, denote y y Y =, y (n 2) y (n 1) then Y = = y y y y = y (n 1) y (n 1) y (n) a n 1 y (n 1) a 1 y a 0 y y y = AY y (n 2) a 0 a 1 a 2 a n 1 y (n 1) Solve the linear differential equation We get the expression of y Ex Y =Y Ex Example 3 (p319 in 7th ed) Briefly describe the process If we have a system consisting of higher order equations, we can similarly change the system to a linear differential system, then solve it (cf p320 in 7th ed) Homework Sect 62 1bf, 2c, 3, Solve y = 4y 45

46 63 Diagonalization Def: A matrix A R n n is diagonalizable iff A is similar to a diagonal matrix D In other words, there is a nonsingular matrix X such that X 1 AX = D We have seen the power of diagonalization in solving linear differential equations More will be explored here What type of matrices A can be diagonalized? Ex Not all n n matrices are diagonalizable [ Thm 64 If A has n distinct eigenvectors λ 1,, λ n with corresponding eigenvectors x 1,, x n, then x 1,, x n are linearly independent By the next [ theorem, Distinct eigenvalues = Diagonalizable The converse is NOT 1 0 true (eg ) 0 1 Thm 65 A R n n is diagonalizable iff A has n linearly independent eigenvectors (So we get a basis of R n consisting of eigenvectors of A) A is diagonalizable X 1 AX = D for a diagonal matrix D = diag(λ 1,, λ n ) AX = XD, ie A[x 1,, x n = [x 1,, x n diag(λ 1,, λ n ) [Ax 1,, Ax n = [λ 1 x 1,, λ n x n the columns of X are eigenvectors, each corresponding to one diagonal entry (eigenvalue) of D This tells us how to diagonalize a matrix A (provided A is diagonalizable) [ [ Ex has two eigenvectors (corresponding to eigenvalue 3) and (corresponding to eigenvalue 1) So [ [ [ = Ex Find eigenvectors to diagonalize the matrix [ Powers and exponentials of diagonalizable matrices: 46 [ [ 1 1

47 1 Power: X 1 AX = D A = XDX 1 A k = XD k X 1 The last equality holds for every integer k 2 Def: (Exponential) The power series of e x is e x = 1 + x + 1 2! x ! x3 + Similarly, we define the matrix exponential by e A = I + A + 1 2! A ! A3 + For a diagonal matrix D = λ 1, λ n e D = I + D + 1 2! D ! D3 + = e λ 1 e λn If A = XDX 1 then A k = XD k X 1 and e A = I + A + 1 2! A ! A3 + = I + XDX ! XD2 X ! XD3 X 1 + = X(I + D + 1 2! D ! D3 + )X 1 = X e λ 1 e λn [ 1 1 Ex Compute e A for A = 1 1 X 1 = Xe D X 1 The solution for the initial value problem Y = AY, Y(0) = Y 0 is: Y = e At Y 0 Ex The solution of y = ay, y(0) = y 0 is y(t) = e at y 0 We can use the previous formula in Section 62 to deduce it Ex Example 8 (p340 in 7th ed) 47

48 Homework Sect 63 1de, 2de, 3de, 17, 27a (24a in 6th ed) 48

49 Review Session What can elementary row operations do? 1 Check consistency, lead variables and free variables Solve a linear system 2 Find the inverse of a nonsingular matrix: [ A I A 1 [ A I = [ I A 1 3 Compute the determinants: Watch the changes of determinant during elementary row operations 4 Linear independence: A set of vectors {v 1,, v n } is linearly independent iff c 1 v c n v n = 0 has only one solution c 1 = = c n = 0 5 Spanning set: A vector v is in span(v 1,, v n ) iff the system c 1 v c n v n = v is consistent We solve the system to express v as a linear combination of v 1,, v n 6 Basis: {v 1,, v n } R n forms a basis of R n iff det[v 1,, v n 0 7 Use row operations to find bases of the row space (R(A T )), column space (R(A)), and nullspace (N(A)), and to find the dimensions of these spaces 8 Let F = [u 1,, u n and H = [v 1,, v n be two bases of R n The transition matrix from F to H is H 1 F : [ H F = [ v1,, v n u 1,, u n [ I H 1 F 9 Let F = [ u 1,, u n be a basis of R n, and H = [ v 1,, v m be a basis of R m The matrix representation of a linear transformation L : R n R m with respect to bases F and H is: [ H L(F ) = [ v1,, v m L(u 1 ),, L(u n ) [ I m A 10 Let the linear transformation L A : R n R m be defined by L A (x) := Ax for some A R m n The kernel of L A is {x R n Ax = 0} = N(A), which can be done by finding the reduced row echelon form of [A 0 The range of L A is {Ax x R n } = R(A), which can be done by finding the reduced row echelon form of A T 11 The least squares problem Ax = b can be done by solving the normalized system A T Ax = A T b 12 The orthogonal complement S of a subspace S 49

50 13 The eigenspace corresponding to eigenvalue λ of A is N(A λi) So we should solve (A λi)x = 0 14 Find the solutions of linear differential systems (Y = AY) and initial value problems (Y = AY, Y(0) = Y 0 ) 15 Diagonalize a matrix A Compute the powers and exponentials of a diagonalizable matrix A Other key materials 1 Matrix algebra: properties of sum and minus, scalar multiplication, matrix multiplication, power, transpose and inverse 2 Determinant: use row operations or cofactor expansions to compute the determinant 3 Similar matrix: A and B are similar iff B = S 1 AS Similar matrices have the same characteristic polynomial, characteristic equations, and eigenvalues (They may have different eigenvectors) 4 Use scalar product (and the other inner product) to compute u, v, v, the angle and distance between u and v, the scalar projection and the vector projection To check the orthogonality or to compute the orthogonal complement S 5 Vectors u 1,, u k are orthonormal iff U T U = I k for U = [ u 1,, u k When they are orthonormal, the projection of v onto span(u 1,, u k ) is 6 The Gram-Schmidt process: p = v, u 1 u v, u k u k u 1 p 1 x 2 p 1 u 2 p 2 x 3 p 2 u 3 7 Find the characteristic equation, eigenvalues, eigenspaces and eigenvectors of a matrix A 50

51 25 Eigenvalues: char poly and char equation (5), eigenvalues (5), eigenspaces (5), linear differential system (5), initial value problem (5) 20 Gram-Schmidt: G-S process (10), orthonormal basis (5), expression (5) 15 Least Square: solutions (10), distance b Aˆx (5) 15 Matrix representation: transition matrix (5), matrix representation (10) 10 Inverse (10) 10 Linear system (10) 5 Dimension of some spaces in matrices (5) 51