Preliminary Linear Algebra 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 100
Notation for all there exists such that therefore because end of proof (QED) Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 2 / 100
Notation A = B A implies B A B A if and only if (iff) B a B a is an element of the set B A B A is a proper subset of B A B A is a subset of B R n Euclidean n-space opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 3 / 100
Matrix Notation A = [a m n ij ] = m n columns. a 11 a 12 a 1n a 21 a 22 a 2n. is a matrix with m rows and n..... a m1 a m2 a mn The entry in the i th row and j th column of A is a ij. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 4 / 100
Square Matrices and Vectors Matrix m n A is said to be square if m = n. A matrix with one column is called a vector. A matrix with one row is called a row vector. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 5 / 100
Notation In these STAT611 notes, bold uppercase letters are typically used to denote matrices, and bold lowercase letters are typically used to denote vectors. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 6 / 100
Examples 0 or 0 n is a vector of zeros. 1 or 1 n is a vector of ones. I or I n or I n n is the identity matrix. For example, 1 0 0 I 3 = 0 1 0 = diag(1, 1, 1). 0 0 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 7 / 100
Special Types of Square Matrices A square matrix n n A is upper triangular if a ij = 0, i > j. A square matrix n n A is lower triangular if a ij = 0, i < j. A square matrix n n A is diagonal if a ij = 0, i j. Write one example for each of these types of matrices. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 8 / 100
Examples 1 0 4 6 0 2 3 5 Upper triangular 0 0 8 2 0 0 0 1 1 0 0 Lower triangular 2 4 0 3 5 6 4 0 0 Diagonal 0 9 0. 0 0 13 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 9 / 100
Matrix Transpose If m n A = [a ij ], the transpose of A, denoted A, is the matrix n m B = [b ij ], where b ij = a ji, i = 1,..., m; j = 1,..., n. That is, B = A is the matrix whose columns are the rows of A and whose rows are the columns of A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 10 / 100
A Symmetric Matrix A square matrix n n A is symmetric if A = A. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 11 / 100
Examples Find the transpose of [ ] 4 2. 3 7 Provide an example of a symmetric matrix. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 12 / 100
Examples [ ] [ ] 4 2 4 3 =. 3 7 2 7 The matrix 4 2 1 2 0 3 1 3 5 is symmetric. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 13 / 100
Matrix Multiplication Suppose and Then A B m n n k A = [a m n ij ] = B = [b n k ij ] = a 1. = [a (1),..., a (n) ] a m b 1. = C = [c ij = m k b n = [b (1),..., b (k) ]. n a il b lj ] = [c ij = a ib (j) ] l=1 a 1 B = [Ab (1),..., Ab (k) ] =. = a mb n a (l) b l. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 14 / 100 l=1
Matrix Multiplication Suppose and [ ] 1 2 A = 3 4 [ ] 5 6 B =. 7 8 Work out AB using AB = n l=1 a(l) b l. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 15 / 100
Matrix Multiplication [ ] [ ] 1 2 5 6 3 4 7 8 = = = [ ] [ ] 1 2 [5 6] + [7 8] 3 4 [ ] [ ] 5 6 14 16 + 15 18 28 32 [ ] 19 22. 43 50 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 16 / 100
Transpose of a Matrix Product (AB) = B A The transpose of a product is the product of the transposes in reverse order. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 17 / 100
Scalar Multiplication of a Matrix If c R, then c times the matrix A is the matrix whose i, j th element is c times the i, j th element of A; i.e., ca 11 ca 12 ca 1n ca 21 ca 22 ca 2n c m n A = c[a ij ] = [ca ij ] = m n m n....... ca m1 ca m2 ca mn Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 18 / 100
Linear Combination If x 1,..., x n R m and c 1,..., c n R, then n c i x i = c 1 x 1 + + c n x n i=1 is a linear combination (LC) of x 1,..., x n. The coefficients of the LC are c 1,..., c n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 19 / 100
Linear Independence and Linear Dependence A set of vectors x 1,..., x n is linearly independent (LI) if n c i x i = 0 c 1 = = c n = 0. i=1 A set of vectors x 1,..., x n is linearly dependent (LD) if c 1,..., c n not all 0 n c i x i = 0. i=1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 20 / 100
Prove or disprove: any set of vectors that contains the vector 0 is LD. Prove or disprove: The following set of vectors is LI. 1 7 9 5, 4, 6. 3 1 7 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 21 / 100
Suppose x 1 = 0 and x 2,..., x n are any other vectors of the same dimension as x 1. If we take c 1 = 1, c 2 = = c n = 0, then n i=1 c ix i = 0 and c 1, c 2,..., c n are not all zero. Thus, any set of vectors containing 0 is LD. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 22 / 100
If we take c 1 = 2, c 2 = 1, c 3 = 1 then c 1 Thus the set of vectors is LD. 1 7 9 5 + c 2 4 + c 3 6 = 0. 3 1 7 One way to arrive at such solution is to search for a solution to the system of the equations: c 1 + 7c 2 + 9c 3 = 0 5c 1 + 4c 2 6c 3 = 0 3c 1 + c 2 + 7c 3 = 0. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 23 / 100
Fact V1: The nonzero vectors x 1,..., x n are LD x j is a LC of x 1,..., x j 1 for some j {2,..., n}. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 24 / 100
Proof of Fact V1: (= ) Suppose there exist c 1,..., c n such that n i=1 c ix i = 0. Let j = max{i : c i 0}. Since x 1,..., x n are nonzero, j > 1. Then j c i x i = 0 = i=1 = j 1 c i x i = c j x j. i=1 j 1 i=1 c i c j x i = x j. ( =) Suppose x j = j 1 i=1 c ix i, then n i=1 d ix i = 0, where { ci if i < j d i = 1 if i = j 0 if i > j. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 25 / 100
Orthogonality The two vectors x, y are orthogonal to each other if their inner product is zero, i.e., x y = y x = n x i y i = 0. i=1 The length of a vector, also known as its Euclidean norm, is x := x x = n xi 2. i=1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 26 / 100
The vectors x 1,..., x n are mutually orthogonal if x ix j = 0, i j. The vectors x 1,..., x n are mutually orthonormal if x ix j = 0 i j, and x i = 1 i = 1,..., n. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 27 / 100
Write down a set of mutually orthogonal but not mutually orthonormal vectors. Write down a set of mutually orthonormal vectors. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 28 / 100
[ ] [ ] 2 0, are mutually orthogonal but not mutually orthonormal. 0 1 [ ] [ ] 1 0, are mutually orthonormal. 0 1 1 2 1 2, 1 2 1 2 are mutually orthonormal. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 29 / 100
Orthogonal Matrix A square matrix with mutually orthonormal columns is called an orthogonal matrix. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 30 / 100
Show that if Q is orthogonal, then Q Q = I. Show that if Q is orthogonal and x is any vector of appropriate dimension, then Qx = x. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 31 / 100
Q Q = [q i q j], where Q = [q 1,..., q n ]. By orthogonality of Q, q 1,..., q n are mutually orthonormal. Thus, q iq j = 0 i j and q i = 1 i = 1,..., n. Q Q = I. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 32 / 100
Qx = (Qx) Qx = x Q Qx = x Ix = x x = x. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 33 / 100
An orthogonal matrix Q is sometimes called a rotation matrix because if a vector x is premultiplied by Q, the result (Qx) is the vector x rotated n 1 n n to a new position in R n. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 34 / 100
Vector Space A vector space S is a set of vectors that is closed under addition (i.e., if x 1 S, x 2 S, then x 1 + x 2 S) and closed under scalar multiplication (i.e., if c R, x S, then cx S). In other words, c 1 x 1 + c 2 x 2 S c 1, c 2 R; x 1, x 2 S. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 35 / 100
Is {x R n : x = 1} a vector space? Is {x R n : 1 x = 0} a vector space? Is { n m Ax : x R m } a vector space? opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 36 / 100
Suppose y R n, c R and y = 1, then cy = (cy) cy = c 2 y y = c y = c. Thus y {x R n : x = 1} does not imply that cy {x R n : x = 1}. Therefore, this set is not a vector space. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 37 / 100
Let S = {x R n : 1 x = 0}. Suppose c 1, c 2 R and x 1, x 2 S, then 1 (c 1 x 1 + c 2 x 2 ) = c 1 1 x 1 + c 2 1 x 2 = 0. Thus c 1 x 1 + c 2 x 2 S and it follows that S is a vector space. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 38 / 100
Let { S = Ax : x R m}. n m Suppose c 1, c 2 R and y 1, y 2 S. y 1, y 2 S = x 1, x 2 R m y 1 = Ax 1 and y 2 = Ax 2. Thus, c 1 y 1 + c 2 y 2 = c 1 Ax 1 + c 2 Ax 2 = A(c 1 x 1 + c 2 x 2 ). c 1 x 1 + c 2 x 2 R m, c 1 y 1 + c 2 y 2 S. It follows that S is a vector space. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 39 / 100
Generators of a Vector Space A vector space S is said to be generated by a set of vectors x 1,..., x n if x S = x = n c i x i for some c 1,..., c n R. i=1 opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 40 / 100
Span of a Set of Vectors The span of vectors x 1,..., x n is the set of all LC of x 1,..., x n, i.e., { n } span{x 1,..., x n } = c i x i : c 1,..., c n R. span{x 1,..., x n } is the vector space generated by x 1,..., x n. i=1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 41 / 100
Find a set of vectors that generates the space {x R 3 : 1 x = 0}; i.e., find a set of vectors whose span is S = {x R 3 : 1 x = 0}. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 42 / 100
Let x 1 = 1 0, x 2 = 0 1. Note that 1 x 1 = 0 and 1 x 2 = 0. 1 1 Thus, x 1, x 2 S so that span{x 1, x 2 } S. Now suppose y = y 1 S. y 2 y 3 Then 0 = 1 y = y 1 + y 2 + y 3 = y 3 = y 1 y 2 so that y 1 x 1 + y 2 x 2 = y 1 0 + 0 y 1 y 2 = y 2 = y 1 y 2 = y. y 1 y 2 y 1 y 2 y 3 S span{x 1, x 2 }, and S = span{x 1, x 2 }. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 43 / 100
Basis of a Vector Space If a vector space S is generated by LI vectors x 1,..., x n, then x 1,..., x n form a basis for S. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 44 / 100
Fact V2: Suppose a 1,..., a n form a basis for a vector space S. If b 1,..., b k are LI vectors in S, then k n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 45 / 100
Proof of Fact V2: Because a 1,..., a n form a basis for S and b 1 S, b 1 = n i=1 c ia i for some c 1,..., c n R. Thus, a 1,..., a n, b 1 are LD by Fact V1. Again, using V1, we have a j a LC of b 1, a 1,..., a j 1 for some j {1, 2,..., n}. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 46 / 100
Thus, b 1, a 1,..., a j 1, a j+1,..., a n generate S. It follows that b 1, a 1,..., a j 1, a j+1,..., a n, b 2 is a LD set of vectors by V1. Again by V1, one of the vectors b 1, b 2, a 1,..., a j 1, a j+1,..., a n is a LC of the preceding vectors. It is not b 2 b 1,..., b k are LI. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 47 / 100
Thus b 1, b 2 and n 2 of a 1,..., a n generate S. If k > n, we can continue adding b vectors and deleting a vectors to get b 1,..., b n generates S. However, then V1 would imply b 1,..., b n+1 are LD. This contradicts LI of b 1,..., b k k n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 48 / 100
Fact V3: If {a 1,..., a n } and {b 1,..., b k } each provide a basis for a vector space S, then n = k. Proof: By V2, we have k n and n k. k = n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 49 / 100
Dimension of a Vector Space A basis for a vector space is not unique, but the number of vectors in the basis, known as dimension of the vector space, is unique. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 50 / 100
Find dim(s), the dimension of vector space S, for S = {x R 3 : 1 x = 0}. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 51 / 100
As demonstrated previously, 1 0 span 0, 1 = S. 1 1 1 0 1 0 Because 0, 1 is a LI set of vectors, 0, 1 1 1 1 1 forms a basis for S. Thus, dim(s) = 2 (even though dimension of vectors in S is 3). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 52 / 100
Consider the set { 0}. Is this a vector space? If so, what is its n 1 dimension? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 53 / 100
{0} is a vector space because c 1 x 1 + c 2 x 2 = 0 {0} c 1, c 2 R and x 1, x 2 {0}. The set {0} generates the vector space {0}. However, {0} is not a LI set of vectors. dim({0}) = 0. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 54 / 100
Fact V4: Suppose a 1,..., a n are LI vectors in a vector space S with dimension n. Then a 1,..., a n form a basis for S. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 55 / 100
Proof of Fact V4: It suffices to show that a 1,..., a n generate S. Let a denote an arbitrary vector in S. By V2, a 1,..., a n, a are LD. By V1, a = n i=1 c ia i for some c 1,..., c n R. Thus { n } S = c i x i : c 1,..., c n R, and the result follows. i=1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 56 / 100
Fact V5: If a 1,..., a k are LI vectors in an n-dimensional vector space S, then there exists a basis for S that contains a 1,..., a k. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 57 / 100
Proof of Fact V5: k n by V2. If k = n, the result follows from V4. Suppose k < n. Then, there exist a k+1 S such that a 1,..., a k+1 are LI. Because if not, a 1,..., a k would generate S (by V1), and thus be a basis of dimension k < n, which is impossible by V3. Similarly, we can continue to add vectors to {a 1,..., a k+1 } until we have a 1,..., a n LI vectors. The result follows from V4. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 58 / 100
Fact V6: If a 1,..., a k are LI and orthonormal vectors in R n, then there exist a k+1,..., a n such that a 1,..., a n are LI and orthornormal. Proof: HW problem. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 59 / 100
Rank of a Matrix It can be shown that the (maximum) number of LI rows of a matrix m n A is the same as the (maximum) number of LI columns of m n A. This number of LI rows or columns is known as the rank of m n A and is denoted rank( m n A) or r( m n A). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 60 / 100
If r( m n A) = m, m n A is said to have full row rank. If r( m n A) = n, m n A is said to have full column rank. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 61 / 100
Inverse of a Matrix If r( n n A) = n, there exists a matrix n n B such that n n A n n B = n n I. Such a matrix B is called the inverse of A and is denoted A 1. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 62 / 100
Prove that r( n n A) = n n n B n n A n n B = n n I. Prove that n n A n n B = n n I = n n B n n A = n n I Thus AA 1 = A 1 A = I. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 63 / 100
Proof: (= ): The columns of A form a basis for R n by V4. Thus, there exists a LC of columns of n n A that equals e i for all i = 1,..., n, where e i is the i th column of the identity matrix n n I. Let b i denote the coefficients of the LC of the columns of A that yields e i. Then, with B = [b 1,..., b n ], we have AB = [Ab 1,..., Ab n ] = [e 1,..., e n ] = I. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 64 / 100
( =): If n n B n n A n n B = n n I, then the columns of A generate R n x R n, ABx = Ix = x. If the columns of A were LD, then a subset of the columns of A would be LI and also generate R n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 65 / 100
However, such a subset would be a basis for R n and thus must have n elements. Thus, the columns of A must be LI. Hence, r( n n A) = n. A n n n n B = n n I = Columns of n n A are LI = Rows of n n A are LI = Rows of n n A are a basis for R n = n n C n n C n n A = n n I. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 66 / 100
Thus, AB = I = CAB = CI = IB = C = B = C. AA 1 = A 1 A = I Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 67 / 100
Singular / Nonsingular Matrix If r( n n A) = n, n n A is said to be nonsingular. If r( n n A) < n, n n A is said to be singular. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 68 / 100
Inverse of a Nonsingular 2 2 Matrix [ ] 1 [ ] a b d b = 1 ad bc if ad bc 0. c d c a [ ] a b is singular if ad bc = 0. c d Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 69 / 100
Column Space of a Matrix The column space of a matrix m n A, denoted by C(A), is the vector space generated by the columns of m n A; i.e., C(A) = {Ax : x R n }. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 70 / 100
dim(c(a)) = r( m n A) because the largest possible subset of LI columns of A is a basis for C(A). m n Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 71 / 100
Let 1 0 3 A = 2 1 4. 4 2 8 Find r(a). Give a basis for C(A). Characterize C(A). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 72 / 100
3 1 2 4 2 0 1 2 = 3 4 8. Thus, the columns of A are LD and r(a) < 3. c 1 1 2 4 + c 2 0 1 2 = c 1 2c 1 + c 2 4c 1 + 2c 2 = 0 = c 1 = c 2 = 0. r(a) = 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 73 / 100
A basis for C(A) is given by 1 2 4, 0 1 2. x C(A) = x = c 1 1 2 4 + c 2 0 1 2 for some c 1, c 2 R n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 74 / 100
Note c 1 x = 2c 1 + c 2 : c 1, c 2 R = C(A) 4c 1 + 2c 2 is the set of vectors in R 3 where the first component is arbitrary and the third component is twice the second component, i.e., {x R 3 : 2x 2 = x 3 }. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 75 / 100
Result A.1: rank(ab) min{rank(a), rank(b)} Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 76 / 100
Proof of Result A.1: Let b 1,..., b n denote the columns of B so that B = [b 1,..., b n ]. Then AB = [Ab 1,..., Ab n ]. This implies that the columns of AB are in C(A). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 77 / 100
dim(c(a)) is rank(a). There does not exist a set of LI vectors in C(A) with more than rank(a) vectors by Fact V2. It follows that rank(ab) rank(a). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 78 / 100
It remains to show that rank(ab) rank(b). Note that rank(ab) is the same as rank((ab) ) = rank(b A ). Our previous argument shows that rank(b A ) rank(b ) = rank(b). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 79 / 100
Provide an example where rank(ab) < min{rank(a), rank(b)}. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 80 / 100
[ ] [ ] 1 1 1 A =, B =. 1 1 1 Then AB = 0. Therefore, rank(ab) = 0, rank(a) = 1, rank(b) = 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 81 / 100
Result A.2: (a) If A = BC, then C(A) C(B), (b) If C(A) C(B), then there exist C such that A = BC. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 82 / 100
Proof of A.2(a): Suppose x C(A) then y x = Ay. Now A = BC = x = BCy. Thus, z x = Bz (namely, z = Cy). x C(B). We have shown x C(A) = x C(B). C(A) C(B). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 83 / 100
Proof of A.2(b): Let a 1,..., a n denote the columns of A. C(A) C(B) = a 1,..., a n C(B). Let c i be such that Bc i = a i i = 1,..., n. Then denote C = [c 1,..., c n ]. It follows that BC = B[c 1,..., c n ] = [Bc 1,..., Bc n ] = [a 1,..., a n ] = A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 84 / 100
Null Space of a Matrix The null space of a matrix A, denoted N (A) is defined as N (A) = {y : Ay = 0}. Note that N (A) is the set of vectors orthogonal to every row of A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 85 / 100
A vector in N (A) can also be seen as a vector of coefficients corresponding to a LC of the columns of A that is 0. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 86 / 100
Note that if A has dimension m n, then the vectors in C(A) have dimension m and the vectors in N (A) have dimension n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 87 / 100
Is the null space of a matrix m n A a vector space? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 88 / 100
Yes. Suppose x N (A). Then c R, A(cx) = cax = c0 = 0. Thus x N (A) = cx N (A) c R. Suppose x 1, x 2 N (A). Then A(x 1 + x 2 ) = Ax 1 + Ax 2 = 0 + 0 = 0. Thus, x 1, x 2 N (A) = x 1 + x 2 N (A). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 89 / 100
Find the null space of 1 2 2 4 A =. 1 2 3 6 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 90 / 100
y 1 + 2y 2 = 0 2y 1 + 4y 2 = 0 Ay = 0 = y 1 2y 2 = 0 3y 1 + 6y 2 = 0 = y 1 = 2y 2 = N (A) = {y R 2 : y 1 = 2y 2 }. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 91 / 100
Result A.3: rank( m n A) = n N (A) = {0}. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 92 / 100
Proof of Result A.3: Let [a 1,..., a n ] = A. Then y 1 y n Ay = [a 1,..., a n ]. = y 1 a 1 + + y n a n. r( m n A) = n a 1,..., a n are LI Ay = 0 only if y = 0 N (A) = {0}. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 93 / 100
Theorem A.1: If the matrix A is m n with rank r, then dim(n (A)) = n r, or more elegantly, dim(n ( m n A)) + dim(c( m n A)) = n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 94 / 100
Proof of Theorem A.1: Let k = dim(n (A)). Results A.3 covers the case where k = 0. Suppose now that k > 0. Let u 1,..., u k form a basis for N (A). Then Au m n i = 0 i = 1,..., k. By Fact V5, there exist u k+1,..., u n such that u 1,..., u n form a basis for R n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 95 / 100
We will now argue that the n k vectors Au k+1,..., Au n form a basis for C(A). If so, then dim(n (A)) + dim(c(a)) = k + n k = n, i.e., dim(n (A)) = n dim(c(a)) = n r. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 96 / 100
First note that Au i C(A) i = k + 1,..., n. Now note that c k+1 Au k+1 + + c n Au n = 0 = A(c k+1 u k+1 + + c n u n ) = 0 = c k+1 u k+1 + + c n u n N (A) n = c 1,..., c k R c 1 u 1 + + c k u k = c j u j j=k+1 = c 1 u 1 + + c k u k c k+1 u k+1 c n u n = 0 = c 1 = = c n = 0 by LI of u 1,..., u n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 97 / 100
Therefore, Au k+1,..., Au n are LI. Now let U = [u 1,..., u n ]. Because u 1,..., u n are LI and a basis for R n, U 1 UU 1 = I. Let x R n be arbitrary and define z = U 1 x. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 98 / 100
Then Ax = AUU 1 x = AUz = [Au 1,..., Au k, Au k+1,..., Au n ]z = [0,..., 0, Au k+1,..., Au n ]z = z k+1 Au k+1 + + z n Au n. Therefore, any vector in C(A) can be written as a LC of Au k+1,..., Au n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 99 / 100
It follows that Au k+1,..., Au n is a basis for C(A). n k = r and k + r = n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 100 / 100