Math113: Linear Algebra. Beifang Chen

Math3: Linear Algebra Beifang Chen Spring 26

Contents Systems of Linear Equations 3 Systems of Linear Equations 3 Linear Systems 3 2 Geometric Interpretation 3 3 Matrices of Linear Systems 4 4 Elementary Row Operations 5 2 Row Echelon Forms 7 2 Patterns of Row Echelon Forms 7 22 The Row Reduction Algorithm 7 23 Solving Linear Systems 8 3 Other Forms of Linear Systems 3 Vectors in R 2 and R 3 32 Vectors in R n 33 Linear Combinations 2 34 Geometric Interpretation of Vectors 3 35 Other Forms of Linear Systems 3 4 Solution Set of a Linear System 6 5 Linear Dependence and Independence 9 2 Matrices and Determinants 2 2 Linear Transformations 2 22 The Standard Matrix of a Linear Transformation 22 23 Invertible Matrices 29 24 Determinants 32 25 Elementary Matrices 35 26 LU-Decomposition 36 27 Interpretation of Determinant 36 3 Vector Spaces 38 3 Vector Spaces 38 32 Some Special Subspaces 39 33 Linear Transformations 4 34 Independent Sets and Bases 4 35 Bases of Null and Column Spaces 42 36 Coordinate Systems 42 36 Dimensions of Vector Spaces 45 37 Rank 46 38 Matrices of Linear Transformations 46 38 Change of Basis 5

4 Eigenvalues and Eigenvectors 53 4 Eigenvalues and Eigenvectors 53 42 How to Find Eigenvectors? 55 43 Change of Bases 57 44 Diagonalization 59 45 Complex Eigenvalues 62 5 Orthogonality 63 5 Inner Product 63 52 Orthogonal Projection to a Single Vector 66 53 Orthogonal Projection 68 54 Gram-Schmidt Process 7 55 Least-Square Method 75 6 Inner Product Space 78 6 Inner Product Spaces 78 62 Diagonalization of Symmetric Matrices 8 63 Quadratic Forms 8 64 Hermitian Forms 84 2

Chapter Systems of Linear Equations Lecture Systems of Linear Equations Linear Systems A linear equation in variables x,,, x n is an equation of the form a x + a 2 + + a n x n b, where the coefficients a, a 2,, a n and the constant b are real or complex numbers, usually known in advance, called constants A system of linear equations (or linear system) is a finite collection of linear equations in same variables For instance, a linear system of m equations in n variables x,,, x n can be symbolically written as a x + a 2 + + a n x n b a 2 x + a 22 + + a 2n x n b 2 () a m x + a m2 + + a mn x n b m A solution of a linear system () is a tuple (s, s 2,, s n ) of numbers that makes each equation a true statement when the values s, s 2,, s n are substituted for x,,, x n respectively The set of all solutions of a linear system is called the solution set of the system Theorem Any system of linear equations has one and only of the following: (a) no solution; (b) unique solution; (c) infinitely many solutions A linear system is called consistent if it has either one solution or infinitely many solutions; otherwise it is called inconsistent, that is, it has no solution 2 Geometric Interpretation The linear system 2x + 3 2x x 2 4 3

has no solution The system has a unique solution The system has infinitely many solutions See Figure 2x + 3 2x 5 x 2 4 2x + 3 4x +2 6 6x +3 9 o x o x o x Figure : No solution, unique solution, and infinitely many solutions Note: A linear equation of two variables represents a straight line in R 2 A linear equation of three variables represents a plane in R 3 In general, a linear equation of n variables represents a hyperplane in the n-dimensional Euclidean space R n 3 Matrices of Linear Systems Definition 2 The augmented matrix of the general linear system () is the table a a 2 a n b a 2 a 22 a 2n b 2 a m a m2 a mn b m and the coefficient matrix of () is a a 2 a n a 2 a 22 a 2n a m a m2 a mn (2) (3) 4

Systems of linear equations can be represented by matrices Operations on equations (for eliminating variables) can be represented by appropriate row operations on the corresponding matrices For example, x + 2x 3 2x 3 +x 3 8 2 2 3 8 3x + +4x 3 7 3 4 7 [Eq 2 2[Eq [Eq 3 3[Eq x + 2x 3 5 +5x 3 2 +x 3 4 ( /5)[Eq 2 ( /2)[Eq 3 x + 2x 3 x 3 2 5x 3 2 R 2 2R R 3 3R 2 5 5 2 4 ( /5)R 2 ( /2)R 3 2 2 5 2 [Eq 3 [Eq 2 R 3 R 2 x + 2x 3 2 x 3 2 2 4x 3 4 4 4 ( /4)[Eq 3 ( /4)R 3 x + 2x 3 2 x 3 2 2 x 3 [Eq + 2[Eq 3 [Eq 2 + [Eq 3 x + 3 3 x 3 R + 2R 3 R 2 + R 3 3 3 [Eq [Eq 2 R R 2 x 3 3 x 3 Lecture 2 4 Elementary Row Operations Definition 3 There are three kinds of elementary row operations on matrices: (a) Adding a multiple of one row to another row; (b) Multiplying all entries in a row by a nonzero constant; (c) Interchanging two rows 5

Definition 4 Two linear systems in same variables are called equivalent if their solution sets are the same Two matrices are called row equivalent if there is a sequence of elementary row operations that transforms one matrix to the other Theorem 5 If the augmented matrices of two linear systems are row equivalent, then the two systems have the same solution set In other words, elementary row operations do not change solution set Proof It is trivial for the row operations (b) and (c) in Definition 3 Consider the row operation (a) in Definition 3 Without loss of generality, we may assume that a multiple of the first row is added to the second row Let us only exhibit the first two rows as follows a a 2 a n b a 2 a 22 a 2n b 2 (4) a m a m2 a mn b m Do the row operation (Row 2) + c(row ) We obtain a a 2 a n b a 2 + ca a 22 + ca 22 a 2n + ca 2n b 2 + cb a m a m2 a mn b m Let (s, s 2,, s n ) be a solution of (4), that is, (5) a i s + a i2 s 2 + + a in s n b i, i m (6) In particular, Multiplying c to both sides of (7), we have Adding (8) and (9), we obtain a s + a 2 + + a n s n b, (7) a 2 s + a 2 + + a 2n s n b 2 (8) ca s + ca 2 + + ca n s n cb (9) (a 2 + ca )s + (a 22 + ca 2 )s 2 + + (a 2n + ca n )s n b 2 + cb () This means that (s, s 2,, s n ) is a solution of (5) Conversely, let (s, s 2,, s n ) be a solution of (5), ie, () is satisfied and the equations of (6) are satisfied except for i 2 Since multiplying c to both sides we have Note that () can be written as a s + a 2 + + a n s n b, c(a s + a 2 + + a n s n ) cb () (a 2 s + a 22 s 2 + + a 2n s n ) + c(a s + a 2 s 2 + + a n s n ) b 2 + cb (2) Subtracting () from (2), we have a 2 s + a 22 s 2 + + a 2n s n b 2 This means that (s, s 2,, s n ) is a solution of (4) Lecture 3 6

2 Row Echelon Forms Definition 2 A matrix is called in row echelon form (REF) if it satisfies the following two conditions: (a) All zero rows are gathered near the bottom; (b) The first nonzero entry of a row, called the leading entry of that row, is ahead of the first nonzero entry of the next row A matrix is called in reduced row echelon form (RREF) if it is in row echelon form, and satisfies two more conditions: (c) The leading entry of every nonzero row is ; (d) Each leading entry is the only nonzero entry in its column A matrix in (reduced) row echelon form is called a (reduced) row echelon matrix Note Sometimes we call row echelon forms just as echelon forms and row echelon matrices as echelon matrices without mentioning the word row 2 Patterns of Row Echelon Forms Typical row echelon matrices are as follows:, where the solid dots represent arbitrary nonzero numbers, and the stars represent arbitrary numbers The following are typical reduced row echelon matrices, Theorem 22 Every matrix is row equivalent to one and only one reduced row echelon matrix Definition 23 If a matrix M is row equivalent to a row echelon matrix E, we say that M has the row echelon form E; if E is further a reduced row echelon matrix, then we say that M has the reduced row echelon form E 22 The Row Reduction Algorithm Definition 24 A pivot position in a matrix A is a location in A that corresponds to a leading entry in a row echelon form of A A pivot column (pivot row) is a column (row) of A that contains a pivot position Algorithm 2 (Row Reduction Algorithm) (The Proof of Theorem 22) () Begin with the leftmost nonzero column, which is a pivot column; the top entry is a pivot position (2) If the entry of the pivot position is zero, select a nonzero entry in the pivot column, interchange the pivot row and the row containing this nonzero entry 7

(3) If the pivot position is nonzero, use row operations to reduce all entries below the pivot position to zero, (and the pivot position to and entries above the pivot position to zero for reduced row echelon form) (4) Cover the pivot row and the rows above it; repeat ()-(3) to the remaining submatrix 23 Solving Linear Systems Example 2 Find all solutions for the linear system x +2 x 3 2x + +4x 3 2 3x +3 +4x 3 Solution Perform the row operations: 2 2 4 2 3 3 4 2 2 2 7 4 2 The system has the unique solution R 2 2R R 3 3R R + R 3 R 2 + 2R 3 2 3 6 3 7 2 x 7 4 x 3 2 2 4 2 Example 22 Solve the linear system x +x 3 x 4 2 x +x 3 +x 4 4x 4 +4x 3 4 2x +2 2x 3 +x 4 3 Solution Do the row operations: 2 4 4 4 4 2 2 2 3 2 2 2 4 4 2 () [ [ () R 2 R R 3 4R R 4 + 2R (/2)R 2 R 3 2R 2 R 4 + (/2)R 2 R + R 2 ( /3)R 2 R 3 R 2 R 2R 2 8

Let c and x 3 c 2 Then x + c c 2, x 4 The solutions of the system are given by x + c c 2 c x 3 c 2 x 4 The variables and x 3 are called free variables Example 23 The linear system with the augmented matrix 2 2 5 2 3 3 4 has no solution because its augmented matrix has the row echelon form () 2 ( 3) [7 2 The last row represents a contradictory equation 2 Theorem 25 A linear system is consistent if and only if the row echelon form of its augmented matrix contains no row of the form [,, b with b Example 24 Solve the linear system whose augmented matrix is 2 A 3 6 3 3 2 7 2 2 4 2 4 6 5 4 9

Solution Do row operations: A R R3 2 3 6 3 3 2 7 2 2 4 2 4 6 5 4 2 2 4 2 2 2 4 5 6 2 2 2 4 2 2 4 5 6 2 2 2 4 3 6 2 2 2 R 2 3R R 4 2R R 2 R 3 R 4 + 2R 2 () [2 () [ [2 2 () 2 R 4 + 3 2 R 3 2 R 3 R 2 R 3 There are three free variables,, x 4 and x 5 Let c, x 4 c 2, x 5 c 3, where c, c 2, c 3 are arbitrary numbers The solutions of the system are given by x 2c c 2 + c 3 c x 3 2 + c 2 2c 3 x 4 c 2 x 5 c 3 x 6 2 Definition 26 A variable in a consistent linear system is called a free variable if its corresponding column in the coefficient matrix is not a pivot column 3 Other Forms of Linear Systems 3 Vectors in R 2 and R 3 Lecture 4 Let R 2 be the set of all tuples (u, u 2 ) of real numbers Each tuple (u, u 2 ) is called a point in R 2 For each point (u, u 2 ) we associate a column [ u u 2 called the vector associated to the point (u, u 2 ) The set of all such vectors are still denoted by R 2 So by a point in R 2 we mean a tuple (x, y)

and, by a vector in R 2 we mean a column Similarly, we denote by R 3 the set of all tuples [ x y (u, u 2, u 3 ) of are real numbers, called points in R 3 We still denote by R 3 the set of all columns u u 2, u 3 called vectors in R 3 For example, (2, 3, ), ( 3,, 2), (,, ) are points in R 3, while are vectors in R 3 2 3, 3 2 Definition 3 The addition, subtraction, and scalar multiplication for vectors in R 2 are defined by [ [ [ u v u ± v ±, u 2 v 2 u 2 ± v 2, [ [ u cu c u 2 cu 2 Similarly, the addition, subtraction, and scalar multiplication for vectors in R 3 are defined by u u 2 u 3 ±, 32 Vectors in R n c v v 2 v 3 u u 2 u 3 u ± v u 2 ± v 2 u 3 ± v 3 Definition 32 Let R n be the set of all tuples (u, u 2,, u n ) of real numbers, called points in R n We still denote by R n the set of all columns u u 2 u n of real numbers, called vectors in R n The vector cu cu 2 cu 3

is called the zero vector in R n, and is denoted by The addition and the scalar multiplication in R n are defined by u v u + v u 2 v 2 u + v 2 u n + c u u 2 u n v n cu cu 2 cu n u n + v n The set R n together with the addition and scalar multiplication is called a vector space Proposition 33 For vectors u, v, w in R n and scalars c, d, () u + v v + u, (2) (u + v) + w u + (v + w), (3) u + u, (4) u + ( u), (5) c(u + v) cu + cv, (6) (c + d)u cu + du (7) c(du) (cd)u (8) u u Subtraction can be defined in R n by 33 Linear Combinations u v u + ( v) Definition 34 A vector v in R n is called a linear combination of vectors v, v 2,, v k in R n if there exist scalars c, c 2,, c k such that v c v + c 2 v 2 + + c k v k The set of all linear combinations of v, v 2,, v k is called the span of v, v 2,, v k, denoted Span {v, v 2,, v k } Example 3 The span of a single nonzero vector in R 3 is a straight line through the origin For instance, Span 3 3t t : t R 2 2t is the straight line in parametric form x 3t t x 3 2t, t R Eliminating the parameter t, the parametric equations reduce to two equations about x,, x 3, { x +3 2 +x 3, 2

Example 32 The span of two linearly independent vectors in R 3 is a plane through the origin For instance, Span 3, 2 s +3t 2s t : s, t R 2 s +2t is the plane through the origin Eliminating the parameters s and t, the parametric equations x s +3t 2s t x 3 s +2t become a single equation 3x 5 7x 3 Example 33 Given vectors in R 3, v, v 2, v 3 ; v (a) The vector v can be expressed as a linear combination of v, v 2, v 3, and it can be expressed in one and only one way as a linear combination v 3v + v 2 + 2v 3 (b) Every vector in R 3 is a linear combination of v, v 2, v 3 (c) The span of {v, v 2, v 3 } is the whole vector space R 3 Example 34 Consider vectors in R 3, v, v 2 2, v 3 5 ; u 2 7 4 5 3, v (a) The vector u can be expressed as linear combinations of v, v 2, v 3 in more than one ways For instance, u v + v 2 + v 3 4v + 3v 2 2v v 2 + 2v 3 (b) The vector v can not be written as a linear combination of v, v 2, v 3 34 Geometric Interpretation of Vectors 35 Other Forms of Linear Systems Definition 35 Let A be an m n matrix and B an n p matrix, a a 2 a n a 2 a 22 a 2n A a m a m2 a mn, B The product (or multiplication) of A and B is an m p matrix c c 2 c p c 2 c 22 c 2p C c m c m2 c mp b b 2 b p b 2 b 22 b 2p b n a n2 b np 3

whose entries c ik are defined by c ik n a ij b jk a i b k + a i2 b 2k + + a in b nk, i m, k p j Proposition 36 Let A be an m n matrix, whose column vectors are denoted by a, a 2,, a n Then for any vector v in R n, v v 2 Av [a, a 2,, a n v a + v 2 a 2 + + v n a n v n Proof Write Then Thus a A Av a a 2 a m a a 2 a n a 2 a 22 a 2n a m a m2 a mn, a 2 a 2 a 22 a m2 a a 2 a n a 2 a 22 a 2n a m a m2 a mn, v v v 2 v n,, a n v v 2 v n a v + a 2 v 2 + + a n v n a 2 v + a 22 v 2 + + a 2n v n a m v + a m2 v 2 + + a mn v n a v a 2 v a m v v a a 2 a m + + v 2 a 2 v 2 a 22 v 2 a m2 v 2 a 2 a 22 a m2 + + + + v n a n a 2n a mn a n v n a 2n v n a mn v n a n a 2n a mn v a + v 2 a 2 + + v n a n Theorem 37 Let A be an m n matrix Then for any vectors u, v in R n and scalar c, (a) A(u + v) Au + Av, 4

(b) A(cu) cau Lecture 5 A general system of linear equations can be written as a x + a 2 + + a n x n b a 2 x + a 22 + + a 2n x n b We introduce the column vectors: a a a m and the coefficient matrix: a m x + a m2 + + a mn x n b m,, a n A a n a mn a a 2 a n a 2 a 22 a 2n a m a m2 a mn ; x x x n ; b [ a, a 2,, a n b b m ; (3) Then the linear system (3) can be expressed by (a) The vector equation form: x a + a 2 + + x n a n b, (b) The matrix equation form: (c) The augmented matrix form: Ax b, [ a, a 2,, a n b Theorem 38 The system Ax b has a solution if and only if b is a linear combination of the column vectors of A Theorem 39 Let A be an m n matrix The following statements are equivalent (a) For each b in R m, the system Ax b has a solution (b) The column vectors of A span R m (c) The matrix A has a pivot position in every row Proof (a) (b) and (c) (a) are obvious (a) (c): Suppose A has no pivot position for at least one row; that is, the reduced row echelon form A of A must have a zero row at bottom Thus the system [A b is inconsistent for any vector b b b m with b m Let r, r 2,, r k be row operations performed on A in the order from r to r k to have transformed A into the row echelon form A We denote by r i the inverse row operations of r i, i k Apply these row operations r k, r k,, r in reverse order from r k to r to transform [A b into [A b (A must have been transformed back to A after these row operations) Then the system [A b is inconsistent with the vector b because [A b is inconsistent and row operations do not change inconsistency This is contradictory to (a) that [A b is consistent for arbitrary vector b 5

Example 35 The following linear system has no solution for some vectors b in R 3 2 +2x 3 +3x 4 b 2x +4 +6x 3 +7x 4 b 2 x + +2x 3 +2x 4 b 3 The row echelon matrix of the coefficient matrix for the system is given by 2 2 3 2 4 6 7 R R 3 2 2 2 4 6 7 R 2 2R 2 2 2 2 3 2 2 2 2 3 R 3 R 2 2 2 2 2 3 2 2 3 Then the following systems have no solution 2 2 2 2 3 R 3 + R 2 2 2 2 4 6 7 2 2 3 R 3 R 2 2 2 2 2 3 2 2 3 2 2 3 2 4 6 7 2 2 R 2 + 2R Thus the original system has no solution for b, b 2 b 3 4 Solution Set of a Linear System A linear system Ax b is called homogeneous if b ; otherwise it is called nonhomogeneous For a nonhomogeneous system Ax b, the system Ax is called its corresponding homogeneous system For a homogeneous system Ax, where A is an m n matrix and is the zero vector in R m, there is always the zero solution x, where is the zero vector in R n This zero solution is also called the trivial solution of Ax ; solutions other than the zero solution are called nontrivial solutions of Ax Theorem 4 The homogeneous system Ax has a nontrivial solution if and only if the system has at least one free variable Moreover, Note 2 For any matrix A, we always have #{pivot positions} + #{free variables} #{variables} #{pivot positions} #{rows}, #{pivot positions} #{columns} Example 4 Find the solution set for the nonhomogeneous linear system x +x 4 +2x 5 2 2x +2 x 3 4x 4 3x 5 3 x +x 3 +3x 4 +x 5 x + +x 3 +x 4 3x 5 3 6

Solution Do row operations to find the reduced row echelon form for the coefficients matrix of the system: 2 2 R 2 + 2R 2 2 4 3 3 R 3 R 3 3 3 R 4 + R 2 2 2 2 2 () [ 2 2 () [2 [ ( )R 2 R 3 + R 2 R 4 + R 2 Let the free variables c, x 4 c 2, x 5 c 3 We have x 2 + c c 2 2c 3 2 x 3 x 4 c 2c 2 + c 3 c 2 + c x 5 c 3 + c 2 2 + c 3 2 Let us write Then v 2 In particular, setting c c 2 c 3,, v, v 2 2, v 3 x v + c v + c 2 v 2 + c 3 v 3, c, c 2, c 3 R x v, is a solution of the nonhomogeneous system, called the particular solution of the nonhomogeneous system However, x c v + c 2 v 2 + c 3 v 3, c, c 2, c 3 R are not solutions of the linear system Ax bb, but solutions of its corresponding homogeneous system Ax, called the homogeneous solutions The homogeneous solutions x v, x v 2, and x v 3 are called the basic solutions of the homogeneous system In fact, the augmented matrix of the corresponding homogeneous system has the reduced row echelon form Set, x 4, x 5, we get the basic solution Set, x 4, x 5, we get the basic solution () [ 2 () [2 [ x v x v 2 2 7

Set, x 4, x 5, we get the basic solution x v 3 The general solution for the nonhomogeneous system can written as x x s + x h, where x s is a special solution and x h are homogeneous solutions Definition 42 For consistent linear system Ax b, any solution is called a particular solution of the system The solutions of its corresponding homogeneous linear system Ax are called the homogeneous solutions of the system Ax b Theorem 43 For a homogeneous linear system Ax, if the m n matrix A has p pivot positions, then (a) The number of free variables of the system is q n p and the system has q basic solutions; (b) Each free variable corresponds to a basic solution obtained as follows: set the free variable equal to and all other free variables equal to Proposition 44 For a homogeneous system, any linear combination of solutions is again a solution of the system, ie, if x u and x v are solutions, then for scalars a and b, the linear combinations are also solutions x au + bv Proposition 45 For a nonhomogeneous system, the difference of any two solutions of the system is a solution of its corresponding homogeneous system, ie, if x u and x v are solutions, then x u v is a solution for its corresponding homogeneous system Theorem 46 (Structure of Solution Set) For a consistent linear system Ax b, if x p is a particular solution and x h are the general solutions of the corresponding homogeneous system Ax, then the general solutions of the system are given by x x p + x h Example 42 The augmented matrix of the nonhomogeneous system x +x 3 +3x 4 +3x 5 +4x 6 2 2x +2 x 3 3x 4 4x 5 5x 6 x + x 5 4x 6 5 x + +x 3 +3x 4 +x 5 x 6 2 has the reduced row echelon form 3 3 2 3 2 8

Then a particular solution x p for the system and the basic solutions of the corresponding homogeneous system are given by 3 x p 3, v, v 2 3, v 3 2 2 The general solution of the system is given by x x p + c v + c 2 v 2 + c 3 v 3 Example 43 For what values of a and b the linear system x +2 +ax 3 2x +b 3x +2 +x 3 has nontrivial solutions Answer: 8a + (3a )(b 4) Example 44 Consider the nonhomogeneous system whose augmented matrix is 2 3 2 2 4 4 5 3 3 6 2 8 5 2 4 6 4 5 Linear Dependence and Independence Lecture 6 Definition 5 Vectors v, v 2,, v p in R n are called linearly independent provided that if a linear combination c v + c 2 v 2 + + c p v p then the coefficients c c 2 c p The vectors v, v 2,, v p are called linearly dependent if there exist constants c, c 2,, c p, not all zero, such that c v + c 2 v 2 + + c p v p Example 5 (a) The vectors,, in R 3 are linearly independent (b) The vector, 2, in R 3 are linearly dependent 5 (c) The vectors,,, 3 in R 3 are linearly dependent 3 (d) The basic solutions of any homogeneous linear system are linearly independent 9

Theorem 52 Any set of vectors containing the zero vector is linearly dependent Theorem 53 Let S {v, v 2,, v p } be a set of vectors in R n If p > n, then S is linearly dependent Proof Let A [v,, v p Then A is an n p matrix, and the equation Ax has n equations in p unknowns Recall that for the matrix A the number of pivot positions plus the number of free variables is equal to p, and the number of pivot positions is at most n Thus, if p > n, there must be some free variables Hence Ax has nontrivial solutions This means that the column vectors of A are linearly dependent Corollary 54 Any set of more than n vectors in R n is linearly dependent Theorem 55 A set S {v, v 2,, v p } (p 2) of vectors is linearly dependent if and only if at least one of the vectors is a linear combination of the other vectors Moreover, if S is linearly dependent and v, then some vector v j (j 2) is a linear combination of the preceding vectors v, v 2,, v j Note 3 The condition v in the above theorem can not be deleted For example, the set { [ [ } v, v 2 is linearly dependent, here v, but the vector v 2 can not be a linear combination of v Theorem 56 The column vectors of a matrix A are linearly independent if and only if the system has the only trivial solution Ax 2

Chapter 2 Matrices and Determinants 2 Linear Transformations Lecture 7 Definition 2 Let X and Y be nonempty sets A function from X to Y is a rule f : X Y such that each element x in X is assigned a unique element y in Y ; the element y is written as y f(x) and is called the image of x under f, and the element x is called the preimage of f(x) Functions are also called maps, or mappings, or transformations Definition 22 A function T : R n R m is called a linear transformation if for any vectors u, v in R n and scalar c, (a) T (u + v) T (u) + T (v), (b) T (cu) ct (u) Example 2 (a) The function T : R 2 R 2, defined by T (x, ) (x + 2, ), is a linear transformation, see Figure 2 x x (,) (,) (,) (,) y (2,) (3,) (,) (,) y Figure 2: The geometric shape under a linear transformation (b) The function T : R 2 R 2, defined by T (x, ) (x + 2, 3x + 4 ), is a linear transformation (c) The function T : R 3 R 2, defined by T (x,, x 3 ) (x + 2 + 3x 3, 3x + 2 + x 3 ), is a linear transformation Example 22 The transformation T : R n R m by T (x) Ax, where A is an m n matrix, is a linear transformation Example 23 The map T : R n R n, defined by T (x) λx, where λ is a constant, is a linear transformation, and is called the dilation by λ 2

Example 24 The refection T : R 2 R 2 about a straight line through the origin is a linear transformation Example 25 The rotation T : R 2 R 2 about an angle θ is a linear transformation, see Figure 22 y T(u+v) PSfrag replacements x O u v u + v T (u) T (v) T (u + v) θ T(u) T(v) t O t t u v u+v x Figure 22: Rotation about an angle θ Proposition 23 Let T : R n R m be a linear transformation Let v, v 2,, v p be vectors in R n and let c, c 2,, c p be scalars We have (a) T (), (b) T (c v + v 2 + + c p v p ) c T (v ) + c 2 T (v 2 ) + + c p T (v p ), (c) If v, v 2,, v p are linearly dependent, then T (v ), T (v 2 ),, T (v p ) are linearly dependent, (d) If T (v ), T (v 2 ),, T (v p ) are linearly independent, then v, v 2,, v p are linearly independent 22 The Standard Matrix of a Linear Transformation Let T : R n R m be a linear transformation Consider the following vectors e, e 2 in the coordinate axis of R n It is clear that any vector x x x n in R n is a linear combination of the vectors e, e 2,, e n as Thus,, e n x x e + e 2 + + x n e n T (x) x T (e ) + T (e 2 ) + + x n T (e n ) This means that T (x) is completely determined by the images T (e ), T (e 2 ),, T (e n ) The ordered set {e, e 2,, e n } is called the standard basis of R n 22

Definition 22 The standard matrix of a linear transformation T : R n R m is the m n matrix [T (e ), T (e 2 ),, T (e n ) [T e, T e 2,, T e n Proposition 222 Let T : R n R m be a linear transformation whose standard matrix is A Then T (x) Ax Proof T (x) T (x e + e 2 + + x n e n ) x T (e ) + T (e 2 ) + + x n T (e n ) [T (e ), T (e 2 ),, T (e n )x Ax Example 22 (a) The linear transformation T : R 2 R 2, T (x, ) (x + 2, 3x + 4 ), can be written as the matrix form ([ ) [ [ [ [ [ x x + 2 2 2 x T x + x 3x + 4 3 2 4 3 4 (b) Let T : R 3 R 2, T (x,, x 3 ) (x + 2 + 3x 3, 3x + 2 + x 3 ) Then T is a linear transformation and its standard matrix is given by x [ T x + 2x 2 + 3x 3 3x x + 2 + x 3 3 [ [ [ 2 3 x + x 3 2 + x 2 3 [ 2 3 3 2 Example 222 Let T θ : R 2 R 2 be a rotation about an angle θ Then [ [ cos θ sin θ T (e ), T (e sin θ 2 ) cos θ x x 3 Thus T θ ([ x ) [ cos θ sin θ sin θ cos θ F igure Lecture 8 [ x Proposition 223 Let f, g : R n R m be linear transformations with the standard matrices A and B, respectively, that is, f(x) Ax, g(x) Bx, x R n Then f ± g : R n R m are linear transformation with standard matrix A ± B, that is, (f ± g)(x) f(x) ± g(x) (A ± B)x, x R n ; and for any scalar a, af : R n R m is a linear transformation with the standard matrix aa, that is, (af)(x) af(x) (aa)x, x R n 23

Example 223 Let f, g : R 3 R 2 be linear transformations defined by (writing in coordinates) f(x,, x 3 ) (x, x 3 ), Writing in vectors, g(x,, x 3 ) (x +, + x 3 ) f x g x 3 x x 3 [ x x 3 [ [ x + + x 3 [ x x 3 x x 3, We then have (f + g) (f g) x x 3 x x 3 [ x + x 3 [ 2x 2 [ x + + x 3 [ 2 2 [ [ x x + x 2 x 3 + x 3 [ [ 2x2 2 2x 3 2 [ x ; x x 3 Hence [ [ 2 2 [ [ 2 2 [ + + [ Definition 224 The composition of a function f : X Y and a function g : Y Z is a function g f : X Z, (g f)(x) g(f(x)), x X Proposition 225 If f : X Y, g : Y Z, h : Z W, then f (g h) (f g) h, ; that is, [f (g h)(x) [(f g) h(x), x X Theorem 226 Let f and g be linear transformations defined by f : R n R m, f(x) Ax and g : R p R n, g(x) Bx 24

Then the composition f g : R p R m is a linear transformation, and (f g)(x) Cx (AB)x Moreover, if A [a ij m n, B [b jk n p, and C [c ik m p, then c ik a i b k + a i2 b 2k + + a in b nk, i m, k p Proof For vectors u, v R p and scalars a, b R, we have (f g)(au + bv) f(g(au + bv)) x f(ag(u) + bg(v)) af(gu) + bf(g(v)) a(f g)(u) + b(f g)(v) Then by the condition of linearity the composition map f g is a linear transformation (Optional for Math3) Let us write the vectors x, y, and z as x x p, y y y 2 y n, z Then the linear transformations f, g, and f g can be written as Equivalently, writing in coordinates, we have z z 2 z m y g(x) Bx, z f(y) Ay, z (f g)(x) Cx y j z i p b jk x k, j n, k n a ij y j j k p c ik x k, i m Substitute y j p k b jkx k into z i n j a ijy j ; we obtain z i n j a ij p b jk x k k p n a ij b jk x k k Comparing the coefficients of the variables x k, we conclude c ik j n a ij b jk, i m, k p j Example 224 Consider the linear transformations S : R 2 R 3, S(x) T : R 3 R 2, T (x) [ [ x x x 3 x x + x +, [ x + + x 3 25

Then S T : R 3 R 3 and T S : R 2 R 2 are both linear transformations, and (S T )(x) ([ ) x + x S(T (x)) S 2 + x 3 x x 3 x + x 3 x + 2 + x 3 x, 2 x 3 (T S)(x) T (S(x)) T x x + x + [ [ [ x 2 2 Hence 2 [ 2 [ [, Proposition 227 Let A be an m n matrix and let B be an n p matrix Write B [b, b 2,, b p Then AB A[b, b 2,, b p [Ab, Ab 2,, Ab p Example 225 Let A and [ 2 AB, B Ab [ 2 3 2 3 [ 2 Write B [b, b 2, b 3 Then [ 2 [ 3 5 [ [ [ 2 5 Ab 2 2 3 8 [ [ [ 3 4 Ab 3 2 5 [ 2 [ 2 3 2 3,,, [ 3 5 4 5 8 5 Example 226 For any real number r, the transformation T n : R n R n defined by T n (x) rx, is a linear transformation, and is called the dilation by r Example 227 Let T : R n R m be a linear transformation with standard matrix A Then T T n T m T The standard matrices of T T n and T m T are equal to ra 26

Theorem 228 (a) If A is an m n matrix, B and C are n p matrices, then A(B + C) AB + AC (b) If A and B are m n matrices, C is an n p matrix, then (A + B)C AC + BC (c) If A is an m n matrix and B is an n p matrix, then for any scalar a, a(ab) (aa)b A(aB) (d) Let I n be n n square identity matrix Then I m A A AI n Definition 229 The transpose of an m n matrix A is the n m matrix A T (j, i)-entry of A, that is, A T a a 2 a n a 2 a 22 a 2n a m a m2 a mn T a a 2 a m a 2 a 22 a m2 a n a 2n a nm whose (i, j)-entry is the Example 228 2 3 4 3 4 5 6 5 6 7 8 T 3 5 2 4 6 3 5 7 4 6 8 Proposition 22 Let A be an m n matrix and B an n p matrix Then (AB) T B T A T Proof Let c ij be the (i, j)-entry of AB Let the (i, k)-entry of B T be b ik and let the (k, j)-entry of AT be a kj Then b ik b ki and a kj a jk Then the (i, j)-entry of (AB) T is c ji n a jk b ki k n k which is the (i, j)-entry of B T A T by the matrix multiplication b ika kj, Theorem 22 Let h : X Y, g : Y Z, f : Z W be functions Then Proof For any x X, (f g) h f (g h) [(f g) h(x) (f g)[h(x) f[g(h(x)) f[(g h)(x) [f (g h)(x) Corollary 222 Let A be an m n matrix, B an n p matrix, and C an p q matrix Then (AB)C and A(BC) are m q matrices, and (AB)C A(BC) Lecture 9 27

Definition 223 A function f : X Y is called one-to-one if distinct elements in X are mapped to distinct elements in Y ; and f is called onto if every element y in Y is the image of some element x in X under f Theorem 224 (Characterization of One-to-One and Onto) Let T : R n R m be a linear transformation with standard matrix A Then (a) T is one-to-one T (x) has the only trivial solution The column vectors of A are linearly independent Every column of A has a pivot position (b) T is onto The column vectors of A span R m Every row of A has a pivot position Example 229 (a) The linear transformation T : R 2 R 3, defined by ([ ) x T 4 [ 2 5 x, x 3 6 2 is one-to-one but not onto The column vectors are linearly independent, but can not span R 3 (b) The linear transformation T 2 : R 3 R 2, defined by x [ T 2 2 3 2 3 4 x 3 is not one-to-one but onto The column vectors are linearly dependent but span R 2 (c) The linear transformation T 3 : R 3 R 3, defined by x 2 3 T 3 2 3 4 x 3 3 4 5 is neither one-to-one nor onto The column vectors are linearly dependent and can not span R 3 (d) The linear transformation T 4 : R 3 R 3, defined by x 2 3 T 4 2 3 2 x 3 3 4 is neither one-to-one nor onto The column vectors are linearly dependent and can not span R 3 (e) The linear transformation T 5 : R 3 R 3, defined by T 4 x x, x 3 x 3 is both one-to-one and onto The column vectors are linearly independent and span R 3 Corollary 225 Let T : R n R m be defined by T (x) Ax (a) If T is one-to-one, then n m (b) If T is onto, then n m x x 3 x x 3 x x 3,,, 28

(c) If T is one-to-one and onto, then m n Proof (a) Since T is one-to-one, the system Ax has only the trivial solution Then the reduced row echelon form of A must be of the form This shows that every column of A has a pivot position Thus n m (b) Since T is onto, the reduced row echelon form B for A can not have zero rows This means that every row of B has a pivot position Note that every pivot positions must be in different columns Hence n m Corollary 226 Let T : R n R n be a linear transformation The following statements are equivalent (a) T is one-to-one (b) T is onto (c) T is one-to-one and onto 23 Invertible Matrices Lecture Definition 23 A function f : X Y is called invertible if there is a function g : Y X such that g f(x) x for all x X, f g(y) y for all y Y If so, the function g is called the inverse of f, and is denoted by g f The function Id : X X, defined by by Id(x) x, is called the identity function Theorem 232 A function f : X Y is invertible if and only if f is one-to-one and onto Proof Assume that f is invertible For two distinct elements u and v of X, if f(u) and f(v) are not distinct, that is, f(u) f(v), then u g(f(u)) g(f(v)) v, a contradiction This means that f is - For any element w of Y, consider the element u g(w) of X We have f(u) f(g(w)) w This shows that f is onto Conversely, assume that f is - and onto For any element y of Y, there exists a unique element x in X such that f(x) y Define g : Y X by g(y) x, where f(x) y Then g is the inverse of f Definition 233 A linear transformation T : R n R m is called invertible if there is a linear transformation S : R m R n such that S T Id n and T S Id m, where Id n is the identity transformation Id n : R n R n defined by Id n (x) x for all x R n The identity transformation Id n from R n to R n is linear and its standard matrix is the n n square identity matrix I n 29

Let T : R n R m be an invertible linear transformation Then T is - and onto; thus m n and T : R n R n Let S : R n R n be the inverse of T If A is the standard matrix of S and B is the standard matrix of T, then AB I n BA Definition 234 A square n n matrix A is called invertible if there is a square n n matrix B such that AB I n BA If A is invertible, the matrix B is called the inverse of A, and is denoted by B A Theorem 235 Let T : R n R n be a linear transformation (a) If T is -, then T is invertible (b) If T is onto, then T is invertible In order to find out whether an n n matrix A is invertible or not, it is to decide whether the matrix equation AX I n has a solution, and if it has a solution, the solution matrix is the inverse of A Write X [x,,, x n Then AX I n is equivalent to solve the systems of linear equations Consider their augmented matrices Ax e, A e 2,, Ax n e n [A e, [A e,, [A e n By row operations the solutions x b, b 2,, x n b n can be obtained by row operation as [A e [I n b, [A e 2 [I n b 2,, [A e n [I n b n Note that the systems have the same coefficient matrix A We may solve these systems simultaneously by reducing the matrix A to the identity matrix I n by row operations, that is, [A I n [I n B, where B [b, b 2,, b n If the reduced row echelon form of [A I n is [I n B, then A is invertible and its inverse is B; otherwise, A is not invertible Example 23 The matrix A 2 2 2 4 is invertible Since 2 2 2 4 2 3 2 2 2 3 2 3 5 2 2 7 2 3 3 2 7 2 3 3, R 2 2R R 3 R R 3 + R 2 ( )R 2 R 2 + 3R 3 R + 2R 3 R R 2 ( )R 3 3

thus Example 232 The matrix 2 2 2 4 2 3 2 3 2 3 4 is not invertible 2 3 2 3 2 3 4 7 2 3 2 3 2 3 4 2 2 8 3 2 3 4 2 2 Lecture Proposition 236 If A and B are n n invertible matrices, then (A T ) (A ) T, (AB) B A R 2 2R R 3 3R ( )R 2 R 3 2R 2 Proof Since taking transposition reverses the order of multiplication, we have A T (A ) T (A A) T I T I, (A ) T A T (AA ) T I T I By definition of inverse matrix, we conclude (A T ) (A ) T Since B A AB B B I AA ABB A By definition of invertible matrix, we conclude (AB) B A Proposition 237 If 2 2 matrix is invertible, then and its inverse is given by [ a b c d ad bc, [ [ a b c d ad bc Proof Assume ad bc, that is, ad bc If ad, then a or d ; b or c The matrix contains either a zero row or zero column; so it is not invertible If ad, then a, b, c, and d are all nonzero Thus a/b c/d This means that the two columns are linearly dependent So the matrix is not invertible Assume ad bc ; we have [ [ a b c d d c b a d c [ ad bc ad bc b a [ ad bc 3

Theorem 238 (Characterization of Invertible Matrices) Let A be an n n square matrix Then (a) A is invertible (b) The reduced row echelon form of A is the identity matrix I n (c) A has n pivot positions (d) Ax has the only trivial solution (e) The column vectors of A are linearly independent (f) The linear transformation T (x) Ax is one-to-one (g) The equation Ax b has at least one solution for any b R n (h) The column vectors of A span R n (i) The linear transformation T (x) Ax from R n to R n is onto (j) There is an n n square matrix B such that BA I n (k) There is an n n square matrix B such that AB I n (l) A τ is invertible Proof (a) (b) (c) (d) (e) (f) (g) (h) (i) (l) (T is invertible) (a) Let S(x) Bx Then (j) (T is one-to-one) (S is the inverse of T ) (B is the inverse of A) (k); (k) (j) is similar 24 Determinants Lecture 2 Let T : R 2 R 2 be a linear transformation with the standard matrix [ a a 2 A a 2 a 22 Recall that the determinant of a 2 2 matrix [ a a 2 a 2 a 22 is defined by [ a a 2 det a a 2 a 2 a 22 a 2 a 22 a a 22 a 2 a 2 Let A [a ij be an n n square matrix For a fixed (i, j), where i m and j n, let A ij (n ) (n ) submatrix of A obtained by deleting the ith row and jth column of A Definition 24 Let A [a ij be an n n square matrix The determinant, denoted det A, is inductively defined by det A a det A a 2 det A 2 + + ( ) n+ a n det A n a det A a 2 det A 2 + + ( ) n+ a n det A n Theorem 242 Let A [a ij be an n n matrix The (i, j)-cofactor of A ( i, j n) is the number Then C ij ( ) i+j det A ij det A a C + a 2 C 2 + + a n C n a C + a 2 C 2 + + a n C n 32

Example 24 For any 3 3 matrix, a a 2 a 3 a 2 a 22 a 23 a 3 a 32 a 33 a a 22 a 33 + a 2 a 23 a 3 + a 3 a 2 a 32 a a 23 a 32 a 2 a 2 a 33 a 3 a 22 a 3 Theorem 243 (Cofactor Expansion Formula) For any n n square matrix A [a ij, In other words, Proposition 244 det A a i C i + a i2 C i2 + + a in C in a j C j + a 2j C 2j + + a nj C nj det A T det A a a 2 a n a 22 a 2n a a 22 a nn a nn Theorem 245 Determinant satisfies the following properties (a) Adding a multiple of one row (column) to another row (column) does not change the determinant (b) Interchanging two rows (columns) changes the sign of the determinant (c) If two rows (columns) are the same, then the determinant is zero (d) If one row (column) of A is multiplied by a scalar γ to produce a matrix B, then det B γ det A Proof Consider n n matrices We proceed induction on n (a) For n 2, we have a a 2 a 2 + ca a 22 + ca 2 a (a 22 + ca 2 ) a 2 (a 2 + ca ) a a 22 a 2 a 2 a a 2 a 2 a 22 ; a + ca 2 a 2 + ca 22 a 2 a 22 (a + ca 2 )a 22 (a 2 + ca 22 )a 2 a a 22 a 2 a 2 a a 2 a 2 a 22 Suppose it is true for all (n ) (n ) matrices Consider the case R j + cr i We divide the case into two subcases: () i and (2) i 2 For i, we have Theorem 246 (The Algorithm for Determinant) Let A [a ij be an n n matrix If a a 2 a n interschanging rows a a 2 a 22 a 2n a 2 adding a multiple of a n a n2 a nn one row to another a n Then det A ( ) k a a 2 a n, where k is the number times of interchanging two rows 33

Example 242 The determinant 2 4 2 3 5 3 4 8 2 4 3 2 4 Theorem 247 Let A be an n n matrix and let be the cofactor of the entry (i, j) Then 2 4 3 2 4 C ij ( ) i+j det A ij n a ik C jk a i C j + a i2 C j2 + + a in C jn k 2 4 3 2 { det A if i j if i j 2 Proof For i j, it follows from the cofactor expansion formula For i j, let us verify for 3 3 matrices For instance, for i and j 3, we have a a C 3 + a 2 C 32 + a 3 C 33 a 2 a 3 a 22 a 23 a 2 a a 3 a 2 a 23 + a 3 a a 2 a 2 a 22 a a 2 a 3 a 2 a 22 a 23 a a 2 a 3 Definition 248 For an n n matrix A [a ij, the adjoint matrix adj A of A is the n n matrix whose (i, j) entry is the (j, i)-cofactor C ji, that is, C C 2 C n C C 2 C n C 2 C 22 C n2 adj A C 2 C 22 C 2n C n C 2n C nn C n C n2 C nn Theorem 249 For any n n matrix A, A adj A (det A)I n Proof We verify for the case of 3 3 matrices a a 2 a 3 a 2 a 22 a 23 C C 2 C 3 C 2 C 22 C 32 a 3 a 32 a 33 C 3 C 23 C 33 3 k a kc k 3 k a kc 2k 3 k a kc 3k 3 k a 2kC k 3 k a 2kC 2k 3 k a 2kC 3k 3 k a 3kC k 3 k a 3kC 2k 3 k a 3kC 3k T det A det A det A Theorem 24 An n n matrix A is invertible if and only if det A 34

25 Elementary Matrices Definition 25 The following n n matrices are called elementary matrices O E(c(i)) c i-th, c ; O E(i, j) O O ith jth ; E(j + c(i)) c ith jth Let I n be the n n identity matrix We have I n I n I n cr i R i R j R j+cr i E(c(i)), E(i, j), E(j + c(i)) Proposition 252 Let A be an m n matrix Then (a) E(c(i))A is the matrix obtained from A by multiplying c to the ith row (b) E(i, j)a is the matrix obtained from A by interchanging the ith row and the jth row (c) E(j + c(i))a is the matrix obtained from A by adding the c multiple of the ith row to the jth row Proposition 253 Elementary matrices are invertible Moreover, E(c(i)) E( c (i)), E(i, j) (j, i), E(j + c(i)) E(j c(i)) Theorem 254 A square matrix A is invertible if and only if it can be written as product A E E 2 E p, of some elementary matrices E, E 2,, E p 35

Theorem 255 A square matrix A is invertible if and only if det A Proposition 256 Let A and B n n matrices Then det AB det A det B Proof Case : det A Then A is not invertible by Theorem 24 We claim that AB is not invertible Otherwise, if AB is invertible, then there is a matric C such that (AB)C I; thus A(BC) I; this means that A is invertible, contradictory to that det A Again by Theorem 24, det(ab) Therefore det AB det A det B Case 2: det A It is easy to check directly that det EB det E det B for any elementary matrix E Let A be written as A E E 2 E k for some elementary matrices E, E 2,, E k Then det AB det(e E 2 E k B) (det E ) det(e 2 E k B) (det E )(det E 2 ) det(e 3 E k B) det E det E 2 det E k det B det A det B 26 LU-Decomposition Theorem 26 Let A be an m n matrix If the A can be reduced to it row echelon form by row operations except switching two rows, then A admits a LU-decomposition, that is, there is a lower triangular m m matrix L and an m n upper triangular matrix U such that A LU Proof Let r, r 2,, r k be the row operations applied to A so that to obtain the row echelon form U of A Let E, E 2,, E k be the elementary matrices corresponding the row operations r, r 2,, r k, respectively Then U is upper triangular and E k E 2 E A U Let L E E2 E Then L is a lower triangular and A LU Example 26 K Lecture 3 27 Interpretation of Determinant Let A [ a a 2 a 2 a 22 be a 2 2 matrix Let e, e 2, e 3 be the standard basis of R 3 Set a a a 2, a 2 a 2 a 22 Then the area of the parallelepiped spanned by the two vectors a and a 2 is the length of the cross product a a 2, that is, a a 2 (a e + a 2 e 2 ) (a 2 e + a 22 e 2 ) a a 22 e 3 a 2 a 22 e 3 a a 22 a 2 a 22 36

Let A [a ij [a, a 2, a 3 be a 3 3 matrix The volume of the parallelotope spanned by the three vectors a, a 2, a 3 is the length of the triple (a a 2 ) a 3, that is, (a a 2 ) a 3 a (a 2 a 3 ) a det A a 2 det A 2 + a 3 det A 3 Example 27 The Vandermonde determinant is a a 2 a 3 a 2 a a 3 a a 2 a 2 2 a 2 3 a 2 2 a 2 a a 2 3 a 3 a a 2 a a 3 a a 2 (a 2 a ) a 3 (a 3 a ) (a 3 a 2 )(a 3 a )(a 2 a ) Example 272 a a 2 a 3 a n a 2 a 2 2 a 2 3 a 2 n j a i ) j>i(a a n a n 2 a n 3 a n n 3 2 2 5 2 4 5 3 6 7 R 2 2R R 4 + 3R R 3 + 4R 2 R 4 R 2 3 2 3 2 4 5 3 3 2 3 2 7 9 7 Theorem 27 (Cramer s Rule) Let A [a, a 2,, a n be an n n invertible matrix with the column vectors a, a 2,, a n Then for any b R n, the unique solution of the system Ax b is given by where x i det A i det A, i, 2,, n A i [a,, a i, b, a i+,, a n, i n Proof If a vector x is the solution of the system, then A[e,, e i, x, e i+,, e n [Ae,, Ae i, Ax, Ae i+,, Ae n [a,, a i, b, a i+,, a n Taking determinant of both sides, we have (det A)x i det A i, i n 37

Chapter 3 Vector Spaces Lecture 4 3 Vector Spaces A vector space is a nonempty set V, whose objects are called vectors, equipped with two operations, called addition and scalar multiplication: For any two vectors u, v in V and a scalar c, there are unique vectors u + v and cu in V such that the following properties are satisfied u + v v + u, 2 (u + v) + w u + (v + w), 3 There is a vector, called the zero vector, such that u + u, 4 For any vector u there is a vector u such that u + ( u) ; 5 c(u + v) cu + cv, 6 (c + d)u cu + du, 7 c(du) (cd)u, 8 u u By definition of vector space it is easy to see that for any vector u and scalar c, For instance, u, c, u ( )u u (3) u + (4) u + (u + ( u)) (2) (u + u) + ( u) (6) ( + )u + ( u) u + ( u) (4) ; c c(u) (7) (c)u u ; u u + u + ( )u u + u + ( )u + ( )u ( )u Example 3 (a) The Euclidean space R n is a vector space under the ordinary addition and scalar multiplication (b) The set P n of all polynomials of degree less than or equal to n is a vector space under the ordinary addition and scalar multiplication of polynomials 38

(c) The set M(m, n) of all m n matrices is a vector space under the ordinary addition and scalar multiplication of matrices (d) The set C[a, b of all continuous functions on the closed interval [a, b is a vector space under the ordinary addition and scalar multiplication of functions Definition 3 Let V be a vector space A nonempty subset H of V is called a subspace of V if H is a vector space under the addition and scalar multiplication of V If so, for any vectors u, v H and scalar c, we have u + v H, cu H For a nonempty set S of a vector space V, to verify whether S is a subspace of V, it is required to check () whether the addition and scalar multiplication are well defined in the given subset S, that is, whether they are closed under the addition and scalar multiplication of V ; (2) whether the eight properties (-8) are satisfied However, the following theorem shows that we only need to check (), that is, to check whether the addition and scalar multiplication are closed in the given subset S Theorem 32 Let V be a vector space A nonempty subset H of V is a subspace if and only if it is closed under the addition and scalar multiplication of V, that is, (a) For any vectors u, v H, we have u + v H, (b) For any scalar c and a vector u H, we have cu H Example 32 (a) For a vector space V, the set {} of the zero vector and the whole space V are subspaces of V ; they are called the trivial subspaces of V (b) For an m n matrix A, the set of solutions of the linear system Ax is a subspace of R n However, if b, the set of solutions of the system Ax b is not a subspace of R n (c) For any vectors v, v 2,, v k in R n, the span Span {v, v 2,, v k } is a subspace of R n (d) For any linear transformation T : R n R m, the image of T is a subspace of R m, and the inverse image is a subspace of R n 32 Some Special Subspaces T (R n ) {T (x) : x R n } T () {x R n : T (x) } Lecture 5 Let A be an m n matrix The null space of A, denoted by Nul A, is the space of solutions of the linear system Ax, that is, Nul A {x R n : Ax } The column space of A, denoted by Col A, is the span of the column vectors of A, that is, if A [a, a 2,, a n, then Col A Span {a, a 2,, a n } The row space of A is the span of the row vectors of A, and is denoted by Row A Let T : R n R m, T (x) Ax, be a linear transformation Then Nul A is the set of inverse images of under T and Col A is the image of T, that is, Nul A T () and Col A T (R n ) 39

33 Linear Transformations Let V and W be vector spaces A function T : V W is called a linear transformation if for any vectors u, v in V and scalar c, (a) T (u + v) T (u) + T (v), (b) T (cu) ct (u) The inverse images T () of is called the kernel of T and T (V ) is called the range of T Example 33 (a) Let A is an m m matrix and B an n n matrix The function F : M(m, n) M(m, n), F (X) AXB is a linear transformation For instance, for m n 2, let [ [ 2 2 A, B 3 2 3, X [ x x 3 x 4 Then F : M(2, 2) M(2, 2) is given by [ [ [ [ 2 x x F (X) 2 2 2x + 2x 2 + 4x 3 + 4x 4 x + 3 + 2x 3 + 6x 4 3 x 3 x 4 2 3 2x + 2 + 6x 3 + 6x 4 x + 3 + 3x 3 + 9x 4 (b) The function f : P 2 P 2, defined by is a linear transformation f(a + a t + a 2 t 2 ) (a + a 2 ) + 2a t + (a + a 2 )t 2, Proposition 33 Let T : V W be a linear transformation Then T () is a subspace of V and T (V ) is a subspace of W Moreover, (a) If V is a subspace of V, then T (V ) is a subspace of W ; (b) If W is a subspace of W, then T (W ) is a subspace of V Proof By definition of subspaces Theorem 332 Let T : V W be a linear transformation Let v, v 2,, v k V (a) If v, v 2,, v k are linearly independent, then T (v ), T (v 2 ),, T (v k ) are linearly independent; (b) If T (v ), T (v 2 ),, T (v k ) are linearly independent, then v, v 2,, v k are linearly independent 34 Independent Sets and Bases Definition 34 Vectors v, v 2,, v p of a vector space V are called linearly independent if, whenever there are constants c, c 2,, c p such that we have c v + c 2 v 2 + + c p v p, c c 2 c p The vectors v, v 2,, v p are called linearly dependent if there exist constants c, c 2,, c p, not all zero, such that c v + c 2 v 2 + + c p v p 4

Theorem 342 Let S {v, v 2,, v k } be a subset of independent vectors in a vector space V If a vector v can be written in two linear combinations of v, v 2,, v k, say, v c v + c 2 v 2 + + c k v k d v + d 2 v 2 + + d k v k, then c d, c 2 d 2,, c k d k Proof If (c, c 2,, c k ) (d, d 2,, d k ), then one of the entries in (c d, c 2 d 2,, c k d k ) is nonzero, and (c d )v + (c 2 d 2 )v 2 + + (c k d k )v k This means that the vectors v, v 2,, v k are linearly dependent This is a contradiction Theorem 343 Vectors v, v 2,, v p with v and p 2, are linearly dependent if and only if some v j with j 2, is a linear combination of the proceeding vectors v, v 2,, v j Proof Since the vectors v, v 2,, v p are linearly dependent, there are constants c, c 2,, c p, not all zero, such that c v + c 2 v 2 + + c p v p Let c j be the last nonzero constant among the ordered constants c, c 2,, c p Then v j can be written as ( v j c ) ( v + c ) ( 2 v 2 + + c ) j v j c j c j c j Note 4 The condition v can not be omitted For instance, the set {, v 2 } (v 2 ) is a dependent set, but v 2 is not a linear combination of the zero vector v Definition 344 Let H be a subspace of a vector space V An ordered set B {v, v 2,, v p } of vectors in V is called a basis for H if (a) B is a linearly independent set, and (b) B spans H, that is, H Span {v, v 2,, v p } Proposition 345 (Spanning Theorem) Let H be a nonzero subspace of a vector space V and H Span {v, v 2,, v p } (a) If some v k is a linear combination of the other vectors of {v, v 2,, v p }, then H Span {v,, v k, v k+,, v p } (b) If H {}, then some subsets of S {v, v 2,, v p } are bases for H Proof It is clear that Span {v,, v k, v k+,, v p } is contained in H Write v k c v + + c k v k + c k+ v k+ + + c p v p Then for any vector v a v + + a k v k + + a p v p in H, we have v (a + a k c )v + + (a k + a k c k )v k +(a k+ + a k c k+ )v k+ + + (a p + a k c p )v p This means that v is contained in Span {v,, v k, v k+,, v p } 4