Linear Algebra, Summer 2011, pt. 2

Transcription

1 Linear Algebra, Summer 2, pt. 2 June 8, 2 Contents Inverses. 2 Vector Spaces Examples of vector spaces The column space The null space The row space and left null space Linear independence, basis, and dimension An aside Rank, linear independence Span, basis Dimension Finding the dimension and a basis for the four fundamental subspaces The previous section, compressed to a page Putting it all together: Solving Ax = b. 22 Inverses. We ve already seen how to invert an elementary matrix, but it was only happy coincidence that it was so easy. However, there is an easy algorithm (easy to remember, sort of a pain to calculate) to find an inverse.

2 Let s suppose for concreteness that A = It will turn out that A is invertible (remember that this is not always true!), and so whatever 3 3 matrix A is, it must be the case that AA = I. Recalling how we multiply matrices a column at a time, this means that A times the first column of A must be (,, ) T. That is to say, we wish to find an x such that Ax = (,, ) T. Then x will be the first column of A. Similarly, to find y, the second column of A, we will solve Ay = (,, ) T. We perform a similar calculation to find the third column of A. The fastest way to do this in practice is simply to solve all three equations at the same time. That is to say, we set up the augmented matrix [A I], then row reduce and back substitute. We will be left with [I A ], assuming A is invertible (otherwise, row reduction will go wrong at some point). Using the A above, we have [A I] = Row reducing, then back substituting (this process is called Gauss-Jordan elimination), we find 2 [I A ] =. 2. We also investigate what happens when this goes wrong. Suppose A = Then row reduction breaks down at 2. Then we conclude that the matrix A was singular. 2

3 2 Vector Spaces. Much of what we ve done so far has been working up the foundations of using matrices as great calculating tools and ways of storing information. Vector spaces hold an important theoretical spot in mathematics. One way to approach them is to realize that many results we ll prove rely only on the following two ingredients:. We have a set V, meaning just a collection of objects with no special a priori relations between them. A helpful V to keep in mind is the collection of vectors in R 2, though we ll create a collection of other vector spaces. 2. We will then define two relations on the set V, called addition and scalar multiplication. These two operations have to obey a certain number of properties, which should be familiar, since vectors in R 2 satisfy them. Properties of vector spaces. (a) Addition is associative. (b) Addition is commutative. (c) There exists an identity element of addition. (d) Every vector has an additive inverse. (e) Scalar multiplication distributes over vector addition. (f) Scalar addition distributes. (a + b)v = av + bv. (g) Scalar multiplication commutes. (h) There exists an identity element of scalar multiplication. Let s gather a stable of examples of vector spaces. 2. Examples of vector spaces. The collection of all n dimensional vectors. We also call this space R n. The scalars are real numbers, and vector addition is, well, vector addition. 3

4 All 5 3 matrices. It is usually helpful to think of vectors as just thin matrices, but in this case we think of a matrix as a vector, with the usual addition and scalar multiplication. All polynomials of degree less than or equal to 3. Notice that we also know how to multiply two polynomials, but we ignore that while we think of this space as a vector space. Also notice that polynomials of degree exactly 3 is not a vector space, since there is no vector. All sequences. Given two sequences (,, 2, 3, 4, 5,...) and (, /2, /4, /8,...), we may add entries pointwise, and perform scalar multiplication similarly. We might call this space R, in analogy to R n All functions on R. Given two functions f(x) and g(x), we can add them h(x) = f(x) + g(x), or multiply by scalars. Thinking geometrically, any plane through the origin in R 3 is also a vector space: we can add any two vectors on that plane, or multiply any vector and stay on that plane. We call this plane a subspace of R 3, and these sorts of objects will be a major object of study. Inasmuch as the definition of a vector space was somewhat esoteric and nitpicky, the requirement to be a subspace is much more real: Definition 2. (Subspace) A subspace of a vector space V is a subset of S of V so that. If v, w are in S, then v + w is in S, and 2. For any scalar c and vector v in S, cv is in S. Intuitively, we say that S is a subspace if it is closed under addition and scalar multiplication. Let s start with some examples of subsets S that are not subspaces, then talk about subsets that are. Example 2. (Not a subspace.) Let S = {(x, y) R 2 y = x 2 }. This is a nice graph that we can draw, but it is not closed under either scalar multiplication or addition. Let s show that thoroughly: we can write an element of S by (t, t 2 ). Checking addition, (t, t 2 ) + (s, s 2 ) = (t + s, t 2 + s 2 ), 4

5 but (unless s or t is zero), (t + s) 2 t 2 + s 2, so we are out of the set. Similarly, we check scalar multiplication (note we don t have to do this, since we already know S is not a subspace!) so long as c and t are not zero. c(t, t 2 ) = (ct, ct 2 ) (ct, c 2 t 2 ), Example 2.2 (Not a subspace.) Wait, you might be saying the problem is that y = x 2 is curved, but subspaces should be flat! This is a good point, and true, but we should be careful because not all flat subsets are subspaces: Let S = {(x, y) R 2 : y = 2 x}. Then S is a line in the plane, but one that doesn t go through the origin. Again, we can write a point in S by (t, 2 t), but (t, 2 t) + (s, 2 s) = (t + s, 4 t s) (t + s, 2 t s), and c(t, 2 t) = (ct, 2c ct) (ct, 2 ct). The problem in the above example was really that the origin is not in S. Notice that it must always be, since if v S, then v S, so v v = is in S. Now let s find all the subspaces of R 2. Example 2.3 (Subspaces of R 2.) One subspace of R 2 is just the zero vector. It is closed under vector addition (since + = ), and scalar multiplication (c = ). Now suppose we have one other element in our subspace S, and call it v. Then certainly all scalar multiples of v must be in S. This corresponds to the line with slope v. In fact, it turns out that this is a subspace: by construction it is closed under scalar multiplication, and if we add cv + dv = (c + d)v, the sum of two scalar multiples of v is also a scalar multiple of v. Now suppose we add another vector w that is not on the line determined by v. By the previous argument we will need all the points on the line generated by w, but then also all sums of the form ( v w av + bw = v 2 w 2 5 ) ( a b ).

6 We can see, either geometrically or analytically with some work, that with the proper choice of a and b, we can get to any point on the plane (this relies on the fact that v and w are non-collinear). Hence R 2 has only the subspaces, every line through the origin, and R 2 itself. Moving up a dimension, R 3 will have subspaces, lines through the origin, planes through the origin, and R 3. Example 2.4 (Subspaces of matrix spaces.) The collection D of n n diagonal matrices is a subspace of M n n, the vector space of all n n matrices. We can verify this since adding two diagonal matrices leaves the matrix diagonal, as does multiplying by a scalar. You might also think of M n n as being a wacky way of writing R n2, and the diagonal matrices are the subspace generated by n nonzero entries. More concretely, M 2 2 can be thought of as R 4, and the diagonal matrices are those vectors in R 4 with the second and third entries zero. 2.2 The column space. We will be studying four important subspaces that are generated by a matrix. They are the row space, the column space, the null space, and the left null space. First will be the column space. We begin with a motivating example. Example 2.5 Suppose we have a system of equations with the coefficient matrix 3 3 A = One interesting question to ask is: For what vectors b does Ax = b have a solution? Another way of thinking of this is as follows: given any vector x R 2, A will give us a vector b R 3. What does the collection of all these vectors look like? Our intuition of multiplying matrices tells us that the answer to these questions for this particular A is the collection of linear combinations of columns of A. We call this the column space of A, and denote it by C(A): C(A) = {(3a + 3b, 2a, 3a + 2b) T R 3 : a, b R}. 6

7 Looking above, it is not immediately clear that C(A) is a subspace in general (or even in particular). In order to show that this is a subspace, we must show that if b and c are vectors in C(A), then so is b + c and kb for all real numbers k. But since b C(A), there is a x such that Ax = b, and a y so that Ay = c. Then A(x + y) = Ax + Ay = b + c, so x + y is the desired vector whose image is b + c. Similarly, A(kx) = kax = kb, so kx is the desired vector to show closure under scalar multiplication. We have shown that the column space of an m n matrix A is always a subspace of R m. Now we have a quick answer to when the equation Ax = b has a solution: precisely when b lies in C(A). Notice that if the row echelon form of a square n n matrix has n pivots, then we can solve Ax = b for all b, so the column space is all of R n. We will discuss a strategy for calculating the column space later, but notice that you can always write it down as a linear combination of all the columns of A, but there is interest in getting rid of useless columns: those columns that can be written as a sum of other ones. For example, the column space of A = can be written, without thinking, as C(A) = {a( 2,, 3) T + b(3, 2, 3) T + c(,, ) T + d(2, 2, 2) T : a, b, c, d R}. However, the sharp reader will notice that we do not need to include this last vector (2, 2, 2) T, as we could equally well just have c+2d times (,, ) T. In this sense, the last vector (or, equivalently, the second to last vector) is extraneous, and we ll write our column space slightly more compactly as C(A) = {a( 2,, 3) T + b(3, 2, 3) T + c(,, ) T : a, b, c R}. It is important to see that these two subspaces are actually the same, just we write the second one more compactly. To use words that we define later the three vectors above are linearly independent, and span the subspace C(A). The second way of writing makes it clear that C(A) has dimension 3 (well, makes it clear once we have defined the word dimension!) 7

8 2.3 The null space. The next subspace generated by a matrix A that we will look at is called the null space, and is denoted by N(A). The null space is the collection of solutions x to the equation Ax =. Hence if A is m n, then N(A) is a subspace of R n (contrast this with C(A) which is a subspace of R m ). First let s confirm that N(A) is indeed a subspace: Closure under vector addition. Suppose x, y N(A). By our definition, this means that Ax = and Ay =. We need to show that x + y is also in the null space. But this just requires the following calculation: A(x + y) = Ax + Ay = + =. Hence N(A) is closed under vector addition. Closure under scalar multiplication. Now suppose k R and x N(A). This means that Ax =. We need to show that kx N(A), which means that A(kx) =. This is just the calculation A(kx) = kax = k =. Hence N(A) is closed under vector addition. We close with a quick example: Example 2.6 (The null space.) We return to the matrix A = 2 2, whose column space we calculated earlier. We also noted that twice the third column is equal to the fourth column. This is the same as observing that the vector (,, 2, ) T N(A), since A = = Since N(A) is a subspace, this means all multiples of (,, 2, ) T are also in N(A). Soon we will develop the tools to show that these are all the 8

9 vectors of N(A), though you may be able to convince yourself by looking at the matrix (or seeking to solve Ax = ). In any case, N(A) = {a(,, 2, ) T : a R} 2.4 The row space and left null space. The row space will be the collection of all d such that there exists an x with xa = d. This is equivalent to the column space of A T. More will be mentioned on this later. Similarly, the left null space is the collection of all x such that xa =. This is the nullspace of A T. Summarizing where all these subspaces live, suppose A is an m n matrix: The column space is a collection of m vectors b so that there is an n vector x with Ax = b. The null space is a collection of n vectors x so that Ax =, where is the m vector consisting of all zeros. The row space is a collection of n vectors d such that there is a m vector x with xa = d. The left null space is a collection of m vectors x such that xa =, where is the n vector consisting of all zeros. 3 Linear independence, basis, and dimension. 3. An aside. We now have many balls in the air, and it helps to reflect on those: We are still waiting to have a really complete solution of the equation Ax = b for general m n coefficient matrices A. The strategy for this will be to first check that b is in the column space of A, then we find a particular solution x p so that Ax p = b. Next we notice that we can add elements y N(A) to x p and still have a solution: A(x p + y) = Ax p + Ay = b + = b. 9

10 Thus, information about the column space will let us know for which b s the equation Ax = b is solvable, and information about the null space will let us know, in a precise sense, how many solutions there are. We learned about vector spaces and subspaces, in particular the column space, null space, row space, and left null space. These will give us information about the matrix, but we are still seeking algorithms for finding these subspaces, as well as language for describing them. This section is mainly concerned with that goal (namely, linear independence, basis, and dimension). We will then use this language to accomplish our goals up above. 3.2 Rank, linear independence. Let us use the matrix A = Factoring, we get A = LU = , hence the pivots are in positions (, ), (2, 3) and (3, 4). We call the number of pivots that a matrix has the rank of a matrix. We have just discovered that rank(a) = 3. The rank will be our first attempt to glean information from a matrix by associating it with a number. The second will be the determinant, which gets significantly more attention in most introductions, but the rank is surprisingly informative for a simple calculation. The information it gives we can already see: The rank of a matrix is the number of linearly independent rows.

11 In the matrix A, our last row could also be written as a sum of the first two rows. Let s now finally define the words linearly independent. Note that this is a vector space definition, and applies to all vector spaces, not just R n. Definition 3. (Linear Independence.) We say a collection of vectors v, v 2,..., v n are linearly independent if the equation a v + a 2 v 2 + a n v n = has only the solution a = a 2 = = a n =. We should note that setting each of the weights a j = for j =,..., n will always satisfy the above equation, so we are only concerned with nonzero solutions. Also note that linear independence applies to a collection of vectors rather than a single vector. We give two examples: Example 3. (Linear Dependence.) The matrix A introduced above has linearly dependent rows, since ( 2, 3,, 2) + (2, 3,, 2) + (,, 4, 9) + ( ) (,, 2, 4) = (,,, ). The columns are also linearly dependent (this is not a coincidence!) since the first column plus the second column is equal to the zero vector (in this case the weights (a, a 2, a 3, a 4 ) = (,,, )). Example 3.2 (Linear Independence.) The vectors (,, ), (,, ) and (,, ) are linearly independent. We can see this since the sum has a particularly simple form: a (,, ) + a 2 (,, ) + a 3 (,, ) = (a, a 2, a 3 ). This vector is zero only when a = a 2 = a 3 =. Example 3.3 (Linear dependence of two vectors.) Given two nonzero vectors v and v 2, the linear independence picture is particularly clear: if, with a, a 2, then we have a v + a 2 v 2 =

12 v = cv 2, for c := a 2 /a. Geometrically, this means that if v and v 2 are linearly dependent, then they are parallel: they point in the same direction. A final example that will be important for our four fundamental subspaces: Example 3.4 (Upper triangular matrices.) Let s go back to our matrix A, but in particular the row echelon form of A; U = 2 4. We already noted that U has 3 linearly independent rows: the first, second and third (any collection of vectors that includes the zero vector is automatically linearly dependent!) First note that the columns are linearly dependent, since 3/2 of the first column plus the second column gives the zero vector. However, we claim that U also has three linearly independent columns, the first, third and fourth. To show this, we must show that a 2 + a a = means that a = a 2 = a 3 =. But notice first that a 3 must be, since it is the only number in the third coordinate. Then using back substitution, 2a 2 + 4a 3 = 2a 2 =. Hence a 2 =. Finally, the first coordinate gives us that a =. This example was general: If A is upper triangular, then the collection of nonzero rows is linearly independent, and the collection of columns containing pivots is linearly independent. We can see that this is true in general, since back substitution will always give us that each of these coefficients need to be zero. 2

13 3.3 Span, basis. Now we define the span of a set of vectors. Again, this is a vector space definition, though it is helpful for the intuition to think of it in the context of R n. Definition 3.2 (Span.) The span of a set of vectors v, v 2,..., v n is the set of all linear combinations of the vectors: span{v, v 2,..., v n } = {a v + a 2 v a n v n : a,..., a n R}. Notice that the span of a collection of vectors is a subspace of the vector space they live in. For example, the column space of a matrix is defined as the span of the columns of the matrix. Also, we point out that the span is not picky: it will eat any collection of vectors. We also use the word span as a verb: If S is a subspace and span({v, v 2,..., v n }) = S, then we say that the vectors {v, v 2,..., v n } span S. Example 3.5 (Span is not picky.) We simply note that R 2 = span{(, ), (, )} = span{(, ), (, ), (, ), (, ), (8, 2)}. By not picky, we mean that the first set above is a much more compact way of writing vectors whose span is R 2. The second set is linearly dependent, meaning there are extraneous vectors in it. What happens when we throw out the extraneous vectors? basis. We get a Definition 3.3 (Basis.) Given a subspace S of a vector space V, a basis for S is a collection of vectors {v, v 2,..., v n } such that. The span span({v, v 2,..., v n }) = S, and 2. the vectors {v, v 2,..., v n } are linearly independent. A basis is great because condition tells us every vector x in S can be written as x = a v + a 2 v a n v n. 3

14 Now suppose also that Then by subtracting we find x = b v + b 2 v b n v n. = (a b )v + (a 2 b 2 )v (a n b n )v n. We then invoke condition 2 of a basis to find that (a j b j ) = for j =,..., n. Hence a j = b j for each j. This means that every vector in a subspace can be written uniquely as a linear combination of basis vectors. Condition lets us use the words can be written, and condition 2 gives us the word uniquely. It is important to note that a subspace does not have a unique basis. For example, {(, ), (, )} is a basis for R 2, as is {(, ), (, )}. This leads to a more general observation: Example 3.6 (Bases of R 2.) How do we show that (, ) and (, ) are linearly independent? We will put them as the columns of a matrix A, and try to find a nonzero solution to Ax =. In particular, we are solving x (, ) T + x 2 (, ) T = ( ) ( x x 2 ) = ( We may either observe that through row reduction rank(a) = 2, meaning A has 2 linearly independent rows, and 2 linearly independent columns, or actually solve through back substitution to find that x = x 2 = is the only solution. Now notice that whenever a 2 2 matrix A has rank 2, the columns will be linearly independent. Also recall that being rank 2 means A has two pivots, which means that A will be invertible. Hence A exists, and so Ax = b has a solution for any b, namely A b. Thus there is a linear combination of the columns of A that give b for any b R 2. Which means that the span of the columns is all of R 2. This is a great fact: The columns of any invertible 2 2 matrix form a basis for R 2. The same analysis as above shows that there is nothing special about the number 2. The columns of an invertible n n matrix will be a basis for R n. We will provide through examples how the columns of a singular square matrix are not a basis for R n. ). 4

15 We approach the same example, geometrically. Example 3.7 (Bases of R 2, geometrically.) Notice that any single vector in R 2 is linearly independent, but the span will only be all the points lying on that line through the origin. Conversely, any three vectors v, v 2 and v 3 (assuming no two of the vectors are parallel) will span R 2, but are linearly dependent, since you can draw a path using only the directions v and v 2 to get to v 3. The right number is two:; any two non-collinear vectors in R 2 will span R 2 and be linearly independent. These two examples were the same, just one argument was made analytically, the other geometrically Dimension. Now we have seen that bases do not have unique bases, but one thing is unique: any two bases of a vector space S will have the same number of basis elements. We call this number the dimension of S. In general, when we encounter a subspace or vector space, the two questions we will ask are:. What is the dimension? 2. What is a basis? Knowing the answer to the first question tells you quite a bit about a vector space. Knowing the answer to the second question lets you know everything about a vector space (in fact, we typically define a vector space by just writing down a basis). First, let s prove the assertion we made earlier: Proposition 3. Every basis for a vector space V has the same number of elements. Proof This will be a proof by contradiction. We will assume that we have two different bases for V, and that the bases have a different number of elements, then show that one of them wasn t actually a basis. So let {v, v 2,..., v m } and {w, w 2,..., w n } be two bases for V, with m < n. Since the basis elements are also elements of V, we can write each w j uniquely as a linear combination of the v k s: m w j = a,j v + a 2,j v a m,j v m = a k,j v k. 5 k=

16 Written as a matrix, this says that W = V A, where W is the matrix with columns w j, V is the matrix with columns v j, and A = (a i,j ) is the (unique) coefficient matrix we just wrote down. Now A is m n, so V A is m n, i.e., this matrix corresponds to a system of m linear equations in n unknowns. Then there exists a nonzero solution to V Ax =, hence W x =, hence there is a linear combination of the columns of W giving zero, and so {w, w 2,..., w n } was not actually linearly independent. 3.4 Finding the dimension and a basis for the four fundamental subspaces. As an application of all the definitions we just gave, we give an algorithm for finding first what the dimension of the four fundamental subspaces is, and then finding a basis. We will go through an example, while at the same time computing a concrete example. For that example we let A = Step. Preliminaries. Perform Gauss-Jordan elimination on the matrix (A I). Note that in general, if A is m n, then this identity matrix is m m. Remember also the difference between Gaussian elimination and Gauss-Jordan elimination. Gaussian elimination gives us Proceeding to back substitute (i.e., completing Gauss-Jordan elimination) gives

17 We will denote this process by (A I) (R E), where R stands for reduced row echelon form, and E stands for elimination. Notice that the rows of E correspond exactly to the linear combination of rows of A that give the rows of R. That is to say, 4 times row of A, minus 3 times row 2 will give the first row of R. As a bit of language, we will call the columns that contain pivots pivot columns, and those that don t free columns. We will denote by r the rank of the matrix, so in this case the rank is 2. In general, we will have r pivot columns, and n r free columns (recall that A is m n). Between the matrices A, R and E, we can read off nearly all the information we need. The null space will still take a bit of work. Step 2. Calculating the column space. We can read off both the dimension and find a basis right away: the dimension of C(A) is r, and a basis consists of the columns of A that correspond to the pivot columns of E. Hence, for our example, C(A) has dimension 2, and a basis is given by, 3 4. It is a (possibly misleading) accident that these two columns generate the same subspace as,, which are the two pivot columns of R. This is not true in general! We should say why this is a basis (and why the corresponding columns of R are not). The second question is easier: there s no reason for it to be. As a simple example, consider the matrix ( 2 2 ). 7

18 Its reduced row echelon form is ( and we can see that a basis for the column space for the first is just the span of (, 2) T, while the column space for the second is generated by (, ) T, so the two are not equal. To explain why these columns of A form a basis for C(A), first note that the collection of all columns of A certainly span C(A): this is the definition of column space. However, the columns of A are not necessarily linearly independent. The information we get from R is that ), Rx = if and only if Ax =. Further, each free column of R is a linear combination of the pivot columns, so the pivot columns are the biggest linearly independent collection of columns of R, and hence the biggest linearly independent collection of columns of A. Step 3. Calculating the null space. The dimension of N(A) is easy: each free column corresponds to a basis element for the null space. Hence in general the dimension of the null space is n r. To find the null space, we will have to actually solve Ax =. However, we have already put A into reduced row echelon form, R, so we can solve instead Rx =. What we do is set one of our free variables equal to, and the rest equal to, then solve. This will be our first basis vector. Then we repeat for each free variable to get a basis. Working through an example will make this clear. We are solving 2 2 x x 2 x 3 x 4 x 5 =, 8

19 where the free variables have been underlined. First we set x 2 =, and x 4 = x 5 =. Multiplying out, we get x + 2 x 3 = Hence, x = 2, and x 3 =. Then our first basis vector is 2. We can confirm this by noting that the second column of A is twice the first column. Proceeding, we set x 4 =, and x 2 = x 5 =. Now we multiply out to find: x + x 3 + =. Hence, x = x 3 =, and the second basis vector for N(A) is. Again we can confirm that the sum of the first and third columns of A gives the fourth column of A. Finally, we set x 5 =, and x 2 = x 4 =. Then multiplying out gives: x + 2 x 3 + =, 9

20 so x = 2, x 3 =, and the final basis vector is 2 Hence, a basis for N(A) is given by 2,., 2. Our algorithm guarantees that this collection is linearly independent: if we refer to the above basis by {v, v 2, v 3 }, then 2c c 2 2c 3 c c v + c 2 v 2 + c 3 v 2 = c 2 c 3 c 2. c 3 Hence, to equal, the second coordinate says that c =, the fourth says c 2 =, and the fifth says that c 3 =. From this we conclude that these vectors are linearly independent. They also span the null space, since this is a complete solution to Rx =. Step 4. The row space. Great news! Row operations do not change the row space (contrast this with how row operations do change the column space above). Hence, the dimension of the row space is also r, and a basis is each of the pivot rows of R (though the pivot rows of A also form a basis, it is just often messier). In our example, we get that the dimension of C(A T ) is 2, and a basis is { ( 2 2 ), ( ) }. 2

21 Step 5. The left null space. More great news! Since E keeps track of our row operations, we can take the rows of E corresponding to the zero rows of R to get a basis for the null space. These are exactly the combinations of rows that give zero. The dimension will be m r: the number of rows, minus the number of pivots. In our example, the dimension of the left null space is, and a basis is { ( ) }. 3.5 The previous section, compressed to a page. Suppose we have a matrix A. Then if we take the augmented matrix [ A I ] and use row operations to reduce A to reduced row echelon form, R, then we call the matrix on the right E, for elimination matrix (note the matrix E s relationship to the elimination matrices from chapter ). That is, we use row reduction to go from: [ A I ] [ R E ] Now we can easily read off the rank, r of the matrix A, by counting the pivot variables in R, as well as calculate: Dimension of C(A): Is just the rank, r. Basis for C(A): Is the r columns of A that correspond to the pivot columns of R. Dimension of C(A T ): Is also the rank, r. Basis for C(A T ): These are the first r ows of R (since row operations do not change the row space, and the first r rows are the pivot rows). Dimension of N(A): This is the number of free variables, which is n r. Basis for N(A): We find n r solutions to the system of equations Rx =, by setting one free variable equal to at a time, while leaving the rest equal to zero, and solving. Dimension of N(A T ): Since this is just the null space of A T, which has r pivots and m r free variables, this must have dimension m r. Basis for N(A T ): Take the bottom m r rows of E. 2

22 4 Putting it all together: Solving Ax = b. Finally, given a coefficient matrix A, you can:. Write succinctly for which b s there is a solution (by writing a linear combination of basis vectors). 2. Given a b, say whether Ax = b has a solution. (this is the same as asking if b is in C(A)) 3. Find all solutions to Ax = b, by taking any solution x p, and adding on all vectors x n in the null space (since A(x p + x n ) = b + = b). 4. Give a number summarizing how many solutions there are (this is the dimension of the null space). We go through one example in detail. Example 4. (Solving Ax = b.) Suppose we wish to solve the system of linear equations { x +2y +3z = 2 The coefficient matrix here is x +2y +4z = 3 A = ( We will answer the above questions in order, but first we will find a basis for the column space and null space. If we go through (A I) (R E), we ll get (R E) = ). ( Hence C(A) has dimension 2, and basis {( ) ( )} 3,. 4 However, we also know that every 2 dimensional subspace of R 2 is all of R 2, so C(A) is all of R 2, and has the much nicer basis ). 22

23 {( ) (, )}. Now N(A) will have dimension, and we can just see from A that a nonzero vector generating N(A) (and therefore a basis) is 2.. Ax = b has a solution whenever b C(A) = R 2. Hence, there are no restrictions on b. 2. The vector (2, 3) T is certainly in R 2, so there will be a solution. 3. We notice that a particular solution is x p = (,, ) T, so our complete solution is 2 2t x = + t = t, where t is any real number. 4. Finally, we can say that the solution space of Ax = b has dimension (the dimension of the null space). 23