Spring 2018 UW-Madison Lecture 13: Row and column spaces 1 The column space of a matrix 1.1 Definition The column space of matrix A denoted as Col(A) is the space consisting of all linear combinations of the columns of A. The dimension of Col(A) is called the column rank of the matrix A. Consider an m-by-n matrix A m n = [a 1 a 2...a n ] where each a j is the column of A and it is a m-by-1 column vector i.e. a j R m j = 12...n. Then the column vector set and column vector space are defined as follows respectively 1. Column vector set: S(A) = {a 1 a 2...a n }. 2. Column vector space: Col(A) = {all linear combinations of the columns of A} { } n = w : w = c 1 a 1 +c 2 a 2 + +c n a n = c i a i = span(s(a)) Col(A)isasubspaceofR m spannedbythecolumnvectorsets(a) = {a 1 a 2...a n }. In other words Col(A) is the smallest subspace containing S(A). Proposition 1.1 (key property of column space). Consider the linear system Ax = b x 1 a 1 + x n a n = b. It implies that the linear system Ax = b is solvable (at least one solution) if and only if b Col(A). Several simple cases: 1 Copy right reserved by Yingwei Wang i=1
Spring 2018 UW-Madison 1. The column space of identity matrix is the whole space: Col(I n ) = R n. It implies that the linear system I n x = b is always solvable for any right hand side b. 2. The column space of zero matrix is the zero space: Col(0 n ) = {0}. It implies that the linear system 0 n x = b is solvable if and only if the right hand side b = 0. 1 2 3. A =. Then the column space of A is a line in R 2 4 2 i.e. Col(A) = {} 1 1 span. In other words v Col(A) it can be written as v = c. 2 2 1.2 How to find the bases for the column space? Given a matrix A m n there are three ways to find the bases for Col(A): 1. Perform the Gaussian elimination on [A b] and find the requirement for b according to Section 3 in Lecture note 8. 2. Perform Gaussian elimination on A and figure out which columns of the original matrices are the bases. 3. Perform Gaussian elimination on A T to get the row reduced echelon form. Example 1. Find the bases for the column space of the matrix 1 1 A = 1 2 0 2. 2 3 Solution: I will show three ways to find the base for Col(A). 1. Perform the Gaussian elimination on [A b]: 1 1 b 1 [A b] = 1 2 b 2 0 2 b 3 2 3 b 4 1 1 b 1 R 1 +R 2 R 2 0 1 b 2 +b 1 ( 2)R 1 +R 3 R 3 0 2 b 3 0 1 b 4 2b 1 2 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison 1 1 b 1 ( 1)R 2 R 2 0 1 (b 2 +b 1 ) 0 2 b 3 0 1 b 4 2b 1 1 1 b 1 ( 1)R 2 +R 4 R 4 0 1 (b 2 +b 1 ) 2R 2 +R 3 R 3 0 0 b 3 2(b 2 +b 1 ). 0 0 b 4 2b 1 +(b 2 +b 1 ) In order to guarantee a solution the last two rows should be zeros which means { b3 2(b 2 +b 1 ) = 0 b 4 2b 1 +(b 2 +b 1 ) = 0 { b3 = 2b 1 +2b 2 b 4 = b 1 b 2. (1.1) There are infinitely many solutions to the linear system (1.1). We can choose b 1 b 2 as free variables and b 3 b 4 are pivot variables. Then we have If we choose b 1 = 1b 2 = 0 then we can get b 3 = 2b 4 = 1; If we choose b 1 = 0b 2 = 1 then we can get b 3 = 2b 4 = 1. So we can obtain the standard basis set for Col(A): 1 0 Col(A) = span 0 2 1 2. 1 1 Another way to solve the linear system (1.1) is to write the matrix form b 1 2 2 1 0 b 2 0 1 1 0 1 b 3 =. (1.2) 0 b 4 We can find two linearly independent solution to the linear system (1.2) which are also the bases for the Col(A). 2. Perform Gaussian elimination on A and figure out which columns of the original matrices are the bases. 3 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison We have already known that 1 1 1 1 A = 1 2 0 2 0 1 0 0. 2 3 0 0 We can observe two pivots in the echelon form which indicates that the two columns of A are linearly independent. So we can simply choose the column set as the basis set 1 1 Col(A) = span 1 0 2 2. 2 3 A very important remark: We can not choose the column vectors in the echelon form after Gaussian elimination since they are just the indicators instead of the bases. We have to go back to the original matrix and find the bases. 3. Perform the Gaussian elimination on A T to get the reduced echelon form: 1 1 0 2 A T = 1 2 2 3 ( 1)R 1 +R 2 R 2 1 1 0 2 0 1 2 1 ( 1)R 2 R 2 1 1 0 2 0 1 2 1 R 2 +R 1 R 1 1 0 2 1. 0 1 2 1 The reduced echelon form of A T also gives us the standard basis set for Col(A): 1 0 Col(A) = span 0 2 1 2. 1 1 4 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison 2 The row space of a matrix. The row space of A denoted as Row(A) is the space spanned by all of the rows of A i.e. Row(A) = {all linear combinations of the rows of A}. Recall the definition of transpose in the end of Lecture note 9. A key property: The row space of A is equivalent to the column space of A T i.e. Row(A) = Col(A T ). Question: How to find the bases for Row(A)? Answer: Let B = A T. We can find the bases for Col(B). and then get the bases for Row(A). Example 2. Find the bases for the row space of the matrix 1 1 A = 1 2 0 2. 2 3 Solution: Let us consider the transpose of A: ( ) 1 1 0 2 B = A T =. 1 2 2 3 We will find the bases for Col(A T ) and then back to Row(A). 1. Work on [B b] = [A T b]: 1 1 0 2 [B b] = [A T b1 b] = 1 2 2 3 b 2 ( 1)R 1 +R 2 R 2 1 1 0 2 b1 0 1 2 1 b 2 b 1 ( 1)R 2 R 2 1 1 0 2 b1 0 1 2 1 b 2 +b 1 It follows that there is no requirement for the right hand size b which means the column space of A T is the whole space R 2. So the row space of A is also R 2 i.e. Row(A) = R 2 = span{(10)(01)}. 5 Copy right reserved by Yingwei Wang.
Spring 2018 UW-Madison 2. Work on B = A T : 1 1 0 2 B = A T = 1 2 2 3 ( 1)R 1 +R 2 R 2 1 1 0 2 0 1 2 1 ( 1)R 2 R 2 1 1 0 2 0 1 2 1 R 2 +R 1 R 1 1 0 2 1. 0 1 2 1 It indicates that the first two columns of B are linearly independent and they can be served as the bases for Col(B) i.e. {( ) ( )} 1 1 Col(B) = span. 1 2 It is equivalent to say that the first two rows of A could be served as the bases for Row(A) i.e. Row(A) = span{(11)( 1 2)}. 3. Work on B T = (A T ) T = A: It is easy to know that 1 1 1 1 1 0 B T = A = 1 2 0 2 G.E. 0 1 ( 1)R 2 +R 1 R 1 0 0 0 1 0 0. 2 3 0 0 0 0 It follows that the standard basis for Col(B) is {( ( 1 0. 0) 1)} So we can go back to Row(A) i.e. Row(A) = span{(10)(01)}. In addition Examples 1 and 2 show that the dimensions of Row(A) and Col(A) are exactly the same. Actually it is true for any matrix A m n. 6 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison Definition 2.1. The rank of a matrix A is the dimension of its row space or column space i.e. rank(a) = dim(col(a)) = dim(row(a)). (2.1) For example we have already known that rank(i n ) = n rank(0 n ) = 0 () 1 2 rank = 1 2 4 1 1 rank 1 2 0 2 = 2. 2 3 Furthermore from Examples 1 and 2 we can also find the bases for Null(A) and Null(A T ): It follows that Null(A) = span{0} 2 1 Null(A T ) = span 2 1 1 0. 0 1 dim(row(a)) + dim(null(a)) = n = the number of columns dim(col(a))+dim(null(a T )) = m = the number of rows. 7 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison 3 Four fundamental subspaces 3.1 The useful properties Consider the matrix A m n. The four fundamental subspaces of A are shown as follows: 1. Column space: Col(A) is a subspace of R m and dim(col(a)) = rank(a); 2. Row space: Row(A) = Col(A T ) is a subspace of R n and dim(col(a T )) = rank(a); 3. Null space: Null(A) is a subspace of R n and dim(null(a)) = n rank(a); 4. Leftnull space: Null(A T ) is a subspace of R m and dim(null(a T )) = m rank(a). 5. For any vector x Null(A) and any vector r Row(A) we have x r = 0. In other words we say Null(A) Row(A) A. 6. For any vector x Null(A T ) and any vector c Col(A) we have x c = 0. In other words we say Null(A T ) Col(A) A. 3.2 Given any matrix A how to find the bases for four fundamental subspaces? See Examples 1 2 3 4 5 of Chapter 4.5 in your textbook about how to find the bases for row spaces and column spaces. Here I will show more examples on how to find the bases for four fundamental subspaces Col(A) Row(A) = Col(A T ) Null(A) and Null(A T ). Example 3. Find the dimension and bases for four fundamental subspaces of the matrix 1 2 0 1 A = 1 1 1 1. 2 3 1 2 Solution: 8 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison 1. Column space. I will show three ways to find the bases for Col(A). (a) Work on [A b]: 1 2 0 1 b 1 [A b] = 1 1 1 1 b 2 2 3 1 2 b 3 1 2 0 1 b 1 ( 1)R 1 +R 2 R 2 0 1 1 0 b 2 b 1 ( 2)R 1 +R 3 R 3 0 1 1 0 b 3 2b 1 1 2 0 1 b 1 ( 1)R 2 +R 3 R 3 0 1 1 0 b 2 b 1. 0 0 0 0 b 3 b 1 b 2 In order to guarantee a solution the last row should be a zero row which means b 1 +b 2 b 3 = 0 b 1 pivot b 2 b 3 free 1 1 Col(A) = span 1 0. 0 1 (b) Work on A: 1 2 0 1 1 2 0 1 1 0 2 1 A = 1 1 1 1 0 1 1 0 0 1 1 0. 2 3 1 2 0 0 0 0 0 0 0 0 It indicates that the first two columns of A are linearly independent so they can be chosen as bases: 1 2 Col(A) = span 1 1. 2 3 (c) Work on A T : 1 1 2 A T = 2 1 3 0 1 1 1 1 2 9 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison 1 1 2 ( 1)R 1 +R 4 R 4 0 1 1 ( 2)R 1 +R 2 R 2 0 1 1 0 0 0 1 1 2 ( 1)R 2 R 2 0 1 1 0 1 1 0 0 0 1 0 1 ( 1)R 2 +R 3 R 3 0 1 1 ( 1)R 2 +R 1 R 1 0 0 0. 0 0 0 It follows that the first two rows in the reduced echelon form are the bases for Col(A): 1 0 Col(A) = span 0 1. 1 1 In addition the dimension of Col(A) is 2 i.e. dim(col(a)) = 2. 2. Row space. I will show three ways to find the bases for Row(A) = Col(A T ). (a) Work on [A T b]: 1 1 2 b 1 [A T b] = 2 1 3 b 2 0 1 1 b 3 1 1 2 b 4 1 1 2 b 1 ( 1)R 1 +R 4 R 4 0 1 1 b 2 2b 1 ( 2)R 1 +R 2 R 2 0 1 1 b 3 0 0 0 b 4 b 1 1 1 2 b 1 R 2 +R 3 R 3 0 1 1 b 2 2b 1 0 0 0 b 3 +b 2 2b 1. 0 0 0 b 4 b 1 10 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison In order to guarantee a solution the last two rows should be zeros which means { b3 +b 2 2b 1 = 0 (b) Work on A T : b 4 b 1 = 0 { 2b1 b 2 b 3 = 0 b 1 b 4 = 0 2 1 1 0 1 0 0 1 b 1 b 2 b 3 b 4 1 Col(A T ) = span 2 0 1 0 = 0 0 1 1 0 Row(A) = span{(1201)(0 110)}. 1 1 2 1 0 1 A T = 2 1 3 0 1 1 0 1 1 0 0 0. 1 1 2 0 0 0 It implies that the first two columns of A T are linearly indepedent so they can be served as the bases i.e. 1 1 Col(A T ) = span 2 0 1 1 1 1 Row(A) = span{(1201)(1111)}. (c) Work on (A T ) T = A: 1 2 0 1 1 2 0 1 1 0 2 1 A = 1 1 1 1 0 1 1 0 0 1 1 0. 2 3 1 2 0 0 0 0 0 0 0 0 11 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison It follows that the first two rows in the reduced echelon form are the bases for Col(A T ): 1 0 Col(A T ) = span 0 2 1 1 1 0 Row(A) = span{(1021)(01 10)}. In addition the dimension of Row(A) is 2 i.e. dim(row(a)) = 2. 3. Null space. We need to solve Ax = 0. 1 2 0 1 1 2 0 1 1 0 2 1 A = 1 1 1 1 0 1 1 0 0 1 1 0. 2 3 1 2 0 0 0 0 0 0 0 0 It indicates that x 3 x 4 are free variables and x 1 x 2 are pivot variables so the bases for null space are 2 1 Null(A) = span 1 1 0 0. 0 1 In addition the dimension of Null(A) is 2 i.e. dim(null(a)) = 2. 4. Left null space. We need to solve A T x = 0. 1 1 2 1 0 1 A T = 2 1 3 0 1 1 0 1 1 0 0 0. 1 1 2 0 0 0 12 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison It indicates that x 1 x 2 are pivot variables and x 3 is a free variable so the bases for left null space are 1 Null(A T ) = span 1. 1 In addition the dimension of Null(A T ) is 1 i.e. dim(null(a T )) = 1. Example 4. Find the dimension and bases for four fundamental subspaces of the matrix 0 1 1 1 A = 0 0 0 0. 0 1 0 1 Solution: 1. Column space. I will show three ways to find the bases for Col(A): (a) Work on [A b]: 0 1 1 1 b 1 [A b] = 0 0 0 0 b 2 0 1 0 1 b 3 0 1 1 1 b 1 ( 1)R 1 +R 3 R 3 0 0 0 0 b 2 0 0 1 0 b 3 b 1 0 1 1 1 b 1 R 2 R 3 0 0 1 0 b 3 b 1. 0 0 0 0 b 2 It implies that the only requirement is b 2 = 0. So the column space of A is 1 0 Col(A) = span 0 0. 0 1 13 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison (b) Work on A: 0 1 1 1 0 1 1 1 0 1 0 1 A = 0 0 0 0 0 0 1 0 0 0 1 0. 0 1 0 1 0 0 0 0 0 0 0 0 It indicates that the second and third columns of original matrix A are linearly independent so they can be served as the bases for column space i.e. 1 Col(A) = span 0 1 1 0. 0 (c) Work on A T : 0 0 0 A T = 1 0 1 1 0 0 1 0 1 1 0 0 R 1 R 3 1 0 1 0 0 0 1 0 1 1 0 0 ( 1)R 1 +R 4 R 4 0 0 1 ( 1)R 1 +R 2 R 2 0 0 0 0 0 1 1 0 0 ( 1)R 2 +R 4 R 4 0 0 1 0 0 0. 0 0 0 It implies that the first two rows in the reduced echelon form could be served as the bases i.e. 1 0 Col(A) = span 0 0. 0 1 14 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison In addition the dimension of Col(A) is 2 i.e. dim(col(a)) = 2. 2. Row space. I will show three ways to find the bases for Row(A) = Col(A T ). (a) Work on [A T b]: 0 0 0 b 1 [A T b] = 1 0 1 b 2 1 0 0 b 3 1 0 1 b 4 1 0 0 b 3 R 1 R 3 1 0 1 b 2 0 0 0 b 1 1 0 1 b 4 1 0 0 b 3 ( 1)R 1 +R 4 R 4 0 0 1 b 2 b 3 ( 1)R 1 +R 2 R 2 0 0 0 b 1 0 0 1 b 4 b 3 1 0 0 b 3 ( 1)R 2 +R 4 R 4 0 0 1 b 2 b 3 0 0 0 b 1 0 0 0 b 4 b 2 In order to guarantee a solution the last two rows should be zeros which means (b) Work on A T : b 1 = 0 b 4 b 2 = 0 0 0 Col(A T ) = span 1 0 0 1 1 0 Row(A) = span{(0101)(0010)}. 0 0 0 1 0 0 A T = 1 0 1 1 0 0 0 0 1 0 0 0. 1 0 1 0 0 0 15 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison It implies that the first and third columns of A T are linearly indepedent so they can be served as the bases i.e. 0 0 Col(A T ) = span 1 1 1 0 1 1 Row(A) = span{(0111)(0101)}. (c) Work on (A T ) T = A: 0 1 1 1 0 1 1 1 0 1 0 1 A = 0 0 0 0 0 0 1 0 0 0 1 0. 0 1 0 1 0 0 0 0 0 0 0 0 It implies that the first two rows of the reduced echelon form could be served as the bases for Col(A T ) i.e. 0 0 Col(A T ) = span 1 0 0 1 1 0 Row(A) = span{(0101)(0010)}. In addition the dimension of Row(A) is 2 i.e. dim(row(a)) = 2. 3. Null space. We need to solve Ax = 0. 0 1 1 1 0 1 1 1 0 1 0 1 A = 0 0 0 0 0 0 1 0 0 0 1 0. 0 1 0 1 0 0 0 0 0 0 0 0 It implies that x 2 x 3 are pivot variables and x 1 x 4 are free variables. 1 0 Null(A) = span 0 0 1 0. 0 1 16 Copy right reserved by Yingwei Wang
Spring 2018 UW-Madison In addition the dimension of Null(A) is 2 i.e. dim(null(a)) = 2. 4. Left null space. We need to solve A T x = 0. 0 0 0 1 0 0 A T = 1 0 1 1 0 0 0 0 1 0 0 0. 1 0 1 0 0 0 It implies that x 1 x 3 are pivot variables and x 2 is a free variable. 0 Null(A T ) = span 1. 0 In addition the dimension of Null(A T ) is 1 i.e. dim(null(a)) = 1. 17 Copy right reserved by Yingwei Wang