Chapter 2 Subspaces of R n and Their Dimensions Vector Space R n. R n Definition.. The vector space R n is a set of all n-tuples (called vectors) x x 2 x =., where x, x 2,, x n are real numbers, together with two binary operations, vector addition and scalar multiplication defined as follows: () Vector addition: To every x =. and y = y 2. in Rn, x x 2 x n x n x + y x 2 + y 2 x + y =.. x n + y n (2) Scalar multiplication: To every number k and vector x =., kx kx 2 kx =. kx n y y n x x 2 x n
Ex. Let x = 2 and y 3. Find 2x + y. 3 4 Subtraction: x y x 2 y 2 x y =.. x n y n.2 Geometrical Explanation 2
.3 Properties () Vector addition: For all vectors x, y and z (i) vector addition is associative: (x + y) + z = x + (y + z); (ii) vector addition is commutative: x + y = y + x; (iii) there exists an element (additive identity or origin) =. such that x + = x for every vector x; x x 2 (iv) for every vector x =, there is an element (additive inverse). x n x x 2 x =. x n such that x + ( x) = ( x) + x =. We may write x y for x + ( y). (2) Scalar multiplication: To numbers a, b and vector x (i) scalar multiplication is associative: a(bx) = (ab)x; (ii) x = x for every vector x. (3) Scalar multiplication distributes over vector addition: a(x + y) = ax + ay. (4) Scalar multiplication distributes over addition of scalars: (a + b)x = ax + ay..4 Subspaces of R n Definition.2. Subspaces of R n A subset W of R n is called a subspace of R n if it has the following properties: 3
a. W contains the zero vector in R n. b. W is closed under addition: if w, w 2 are both in W, then so is w + w 2. c. W is closed under scalar multiplication: If w is in W and k is an arbitrary scalar, then kw is in W. Ex. W = {} and W = R n are two trivial subspaces of R n, Ex. Let W = {(x, ) x is a real number}, that is, W is the x-axis of R 2. Then W is a subspace of [ R] 2. x Ex. Is W = { in R y 2 x, y } a subspace of R 2? [ ] x Ex. Is W = { in R y 2 xy } a subspace of R 2? Question: What are all possible subspaces of R 2? Theorem.. Suppose W is a subset of R n. Then W is a subspace of R n iff, for all x, y V, and all scalars a, b, x, y W ax + by W. 4
2 Null and Column spaces of Matrices 2. Homogeneous system Consider the following homogeneous linear system of m equations and n unknowns a x + a 2 x 2 + + a n x n = a 2 x + a 22 x 2 + + a 2n x n = Or where a m x + a m2 x 2 + + a mn x n = Ax =, a a 2 a n a 2 a 22... a 2n A =...... a m a m2... a mn x x 2 is the coefficient matrix and x =.. x n x x 2 Then x =. = = is a solution. Moreover, it x and y are two solutions. x n of the system, so are. x + y 2. kx, where k is any number. Therefore, ax + by are also solutions to the system, where a, b are numbers. ax + by is called a linear combination of x and y. The set of all solutions to the linear system Ax =, is called the Null space of matrix A, denoted by Null(A) or N(A). It is a subspace of R n. It is also called the kernel of A, denoted by Ker(A). 5
2.2 Inhomogeneous System b b 2 b m For b =, the system of linear equation. Ax = b may or may not be compatible. When it is compatible, assume that x is any solution of Ax = b, then any solution x of Ax = b can be written x = x + y, where y is a solution of the homogeneous system Ay =. That is, the solution set of Ax = b is x + N(A). 2.3 Span Definition 2.. A vector b in R n is called a linear combination of the vectors v, v 2,, v k in R n if there are scalars c, c n, c k such that b = c x + c 2 x 2 + + c k x k. Ex. Note that [ ] [ ] [ ] [ ] [ ] x x = + = x x 2 x + x 2 2. [ ] [ ] [ ] x is a linear combination of and. x 2 2 Ex. 3 is a linear combination of and since 3 2 3 = 2 +. 3 Definition 2.2. The set of all linear combinations of v, v 2,, v k is a subspace. It is called the span of v, v 2,, v k, denoted by Span{v, v 2,, v k }. [ ] [ ] Ex. What is the span of and? [ ] [ ] Ex. What is the span of and? [ ] [ ] 3 Ex. What is the span of and? 3 6
2.4 Column space of matrices Let A be am m n matrix and v, v 2,, v n be the column vectors of matrix A. The span of column vectors: span{v, v 2,, v n } is called the column space of matrix A. It is also called the range of A, denoted by R(A). 2.5 Geometrical meaning of Ax = b 3 Linear Independence Definition 3.. Linear independence Let v, v 2,, v k be vectors in a subspace V of R n. The vectors v, v 2,, v k are said to be linearly independent if none of them is the linear combination of others. Otherwise, the vectors are said to be linearly dependent. Ex. [ ] Vectors [ ] 7 v = and v 2 = are linearly dependent. 7 Ex. The columns v, v 2, v 3, v 4 of the matrix 3 3 2 A = 2 6 9 5 3 3 are linearly dependent, since v 2 = 3v. The rows are also linearly dependent since row 3 is 2 times row 2 minus 5 times row. Definition 3.2. Linear relations Consider the vectors v, v 2,, v m in R n. An equation of the form c v + c 2 v 2 + + c m v m = is called a linear relation among the vectors v, v 2,, v m. There is always the trivial relation, with c = c 2 = = c m =. Nontrivial relation may or may not exist among the vectors v, v 2,, v m. In the previous example, we have It has a nontrivial relation. 3v v 2 + v 3 + v 4 =. 7
Theorem 3.. The vectors v, v 2,, v m in R n are linearly dependent iff there is a nontrivial relation among them. Proof. If one of the vectors in v i is a linear combination of the others, we have v i = c v + c 2 v 2 + + c i v i + c i+ v i+ + + c m v m Then, by choosing c i = we have c v + + c i v i + c i v i + c i+ v i+ + c m v m =, where not all of the c i are zero, i.e., there is a nontrivial relation. Conversely, if there is a nontrivial relation with c i, then we can solve for v i : c v + + c i v i + + c m v m =, v i = c c i v + + c i c i v i + c i+ c i v i+ + c m v m =, that is, v i is a linear combination of the others. Ex. Check if the column vectors of matrix 3 4 2 A = 5 2 are linearly dependent. Ex. Determine whether the following vectors are linearly independent: 6 2 2 3 4, 7 8 9, 3 5 7, 4 9 6. 5 25 8
Solution To find the relations among these vectors, we consider the vector equations: 6 2 2 c 3 4 + c 7 2 8 9 + c 3 3 5 7 + c 4 4 9 6 =. 5 25 Or the matrix equation 6 2 2 7 3 4 c 3 8 5 9 c 2 4 9 7 6 c 3 =. c 5 25 4 In other words, we have to find the kernel or the null of A. To do so, we compute rref(a) rref(a) =. Therefore the kernel or the null space of A is {}. There is only trivial relation among the vectors, and they are linearly independent. Since any relation c v + c 2 v 2 + + c m v m = can alternatively be written as the matrix form: c v v 2 v m c 2. =. We can generalize the observation made from the above example. Theorem 3.2. The vectors v, v 2,, v m in R n are linearly independent if and only if Ker v v 2 v m = {}. 9 c 4
or, equivalently, This condition implies that m n. rank v v 2 v m = m. Corollary 3.3. In R n, any n + vectors are linearly dependent. 4 Bases and Dimension Definition 4.. Let v, v 2,, v k be vectors in a subspace V of R n. vectors {v, v 2,, v k } is called a basis of V if The set of (i) v, v 2,, v k are linearly independent; (ii) v, v 2,, v k span V, i.e., every vector in V is a linear combination of the vectors v, v 2,, v k. Examples: In R n, the set of vectors {e, e 2,, e n } is a basis, where.... e =, e 2 =,, e i = and e n =..... It is called the standard basis. There are other bases. Ex. In R 2, the set of vectors {e, e 2 } is a basis, [ ] [ ] e = and e 2 =. 5 The Dimension od a Subspace of R n Lemma 5.. Consider vectors v, v 2,, v p and w, w 2,, w q in a subspace V of R n. If the vectors v i are linearly independent and the vectors w, w 2,, w q span V, then p q.
Proof. Since w, w 2,, w q span V, each v i can be expressed as a linear combination of the vectors v, v 2,, v p : v = a w + a 2 w 2 + + a q w q, v 2 = a 2 w + a 22 w 2 + + a 2q w q,, v p = a p w + a p2 w 2 + + a pq w q. We write each of these equations in matrix form: a a p w w 2 w q. = v,, w w 2 w q. = v p. a q We combine all these equations into one matrix equation: a a p w w q... = v v 2 v p a q a pq Or MA = N. Note that the kernel of A is contained in the kernel of N (since if Ax =, then Nx = MAx = ). But the kernel of N is {} since v, v 2,, v p are linearly independent. Therefore, the kernel of A is {} as well. This implies the rank(a) = p q. Theorem 5.2. All bases of a subspace V of R n consist of the same number of vectors. Proof. Consider two bases v, v 2,, v p and w, w 2,, w q of a subspace V. Since v, v 2,, v p are linearly independent and w, w 2,, w q span V, we have p q. Similarly, since w, w 2,, w q are linearly independent and v, v 2,, v p span V,we deduce q p. Therefore, p = q. Definition 5.. The number of elements in a basis of a finite-dimensional vector space V is called the dimension of V and is denoted by dim V. Theorem 5.3. Let V be a subspace of R n with dim(v ) = m. Then i. We can find at most m linearly independent vectors in V ii. We need at least m vectors to span V. iii. Any m linearly independent vectors in V form a basis. iv. If m vectors span V, then they form a basis of V. a pq
5. Find a basis of the kernel Ex. Find a basis of the kernel of the following matrix, and determine the dimension of the kernel: [ ] 2 3. 2 4 9 5 we can generalize the result of this example: Theorem 5.4. Consider an m n matrix A. Then, dim(ker(a)) = n rank(a). 5.2 Find a basis of the Image Ex. Find a basis of the image of the matrix 2 A = 2 2 2 2, and then determine the dimension of the image. Solution: The image of A is the span of column vectors v, v 2,, v 5 of the matrix A. But the column vectors are linearly dependent (?). To construct a basis of the image(a), we could find a relation among columns of A, express one of the columns as a linear combinations of the others and then omit this vectors as redundant. After repeating this procedure a few times, we would be left with linearly independent vectors that still span Im(A): i.e., we would have a basis of Im(A). We propose a method for find a basis of Im(A) that uses this idea, but presents them in somewhat streamlined form. We find the reduced row-echelon form of A. 2 2 2 2 2 3 A = 2 2 E = rref(a) = We denote the ith columns of A and E by v i and w i, respectively. We have to express some of the v i as linear combinations of the other vectors v j to identify redundant vectors. The corresponding problem for the w i is easily solved by inspection: w 3 = w 2w 2, w 4 = 2w 3w 2.. 2
In general, we can express any column of rref(a) that dees not contain a leading as a linear combination of earlier columns that contain a leading. How is that connecting to the case for v i? It may surprise you that the same relationships hold among the corresponding columns of the matrix A: w 3 = w 2w 2, w 4 = 2w 3w 2. Let us explain why corresponding relations hold among the columns of A and rref(a). Consider a relation c v + c 2 v 2 + + c 5 v 5 = among the columns of A. This relation can be written as c c 2 A. =. c 5 Since A and E have the same kernels (the reduced row-echelon form was defined so that Ax = and Ex = have the same solutions), it follows that or c c 2 E. =, c 5 c w + c 2 w 2 + + c 5 w 5 =. Consider the columns v, v 2 and v 5 of A corresponding to the columns of E containing the leading s. Since w, w 2 and w 5 are linearly independent, so are the vectors v, v 2 and v 5. The vectors v, v 2 and v 5 span the image of A, since any vectorv in the image of A can be expressed as v = c v + c 2 v 2 + + c 5 v 5 = c v + c 2 v 2 + c 3 (v 2v 2 ) + c 4 (2v 3v 2 ) + c 5 v 5. That is, v can be written as a linear combination of v, v 2 and v 5. We have shown that v, v 2 and v 5 for a basis of Im(A) and dim(im(a)) = 3. Definition 5.2. A column of a matrix A is called a pivot column if the corresponding column of rref(a) contains a leading. Theorem 5.5. The pivot columns of a matrix A form a basis of Im(A). 3
Theorem 5.6. For any matrix A, and rank(a) = dim(im(a)). Now consider an m n matrix A. We have dim(ker(a)) = n rank(a). dim(im(a)) = rank(a). Adding these two together, we obtain the following; Theorem 5.7. Rank-Nullity Theorem For any m n matrix A, dim(ker(a)) + dim(im(a)) = n. The dimension of the kernel of A is called the nullity of A: Thus we have nullity(a) = dim(ker(a)). nullity(a) + rank(a) = n. Second method We apply the elementary column transformations 2 2 A = 2 2 2 2 2 2 2 2 2 2 2 2 3 2 2 3 F = rcef(a) =. 2 3 2 3 Im(A) = span,,. 4
5.3 Bases in R n Summary Consider an n n matrix a a 2 a n a 2 a 22... a 2n A =....... a n a n2... a nn The following statement are equivalent: i. A is invertible. ii. The linear system Ax = b has a unique solution x, for all b in R n. iii. rref(a) = I n. iv. rank(a) = n. v. Im(A) = R n. vi. ker(a) = {}. vii. The columns of A form a basis of R n. viii. The columns of A span R n. ix. The columns of A are linearly independent. 5