6.2. Orthogonal Complements and Projections In this section we discuss orthogonal complements and orthogonal projections. The orthogonal complement of a subspace S is the set of all vectors orthgonal to S denoted S. We will show that S is in fact a subspace complementary to S. Then the orthogonal projection onto S is the projection onto S with respect to S. Consider any set of vectors E R n. What happens if we look at all of the vectors orthogonal to each vector in E denoted E (pronounced E perp )? Definition 6.2.. For any E R n define E = { x R n x y = for all y E}. When S is a subspace we call S the orthogonal complement of S. Example 6.2.2. Express the following as the span of a finite set of vectors. (a) 2 (b) 2 Solutions: We leave the conversione between relation form and span form for the reader to verify. (a) x x x 2 = 2 = x = 2 x x x + 2 + x = = span. (b) 2 = x x 2 = x = x x x x = x x + 2 + x = + x = = span. Notice that E is a subspace in these two examples which is not a coincidence. Proposition 6.2.. For any E R n E is a subspace. Proof. We show that E satisfies the three properties of being a subspace. (a) We have E since y = for all y E. Thus E. (b) Suppose x E. Then for all y E ( x + ) y = x y + y = + = since x E. Therefore x + E and E is closed under addition. (c) Suppose x E and c is a scalar. Then for all y E (c x) y = c( x y) = c() = since x E. Therefore c x E and E is closed under scalar multiplication.
2 However if E is an infinte set like a subspace it looks like we have to solve an infinite set of equations to find E. How can we possibly find E if E is a nonzero subspace? We need to realize that there is a lot of redudancy in these equations. For example if x y then x c y for all scalars c. In fact we can reduce the set of equations to just being orthogonal to a basis for S. Proposition 6.2.4. For any v... v k R n (span( v... v k )) = { v... v k }. Proof. Let S = span( v... v k ). First if x S then x v =... x v k = because v... v k S. Thus x { v... v k } so S { v... v k }. On the other hand if x { v... v k } then x v =... x v k = Now for any y S we can write y = c v + + c k v k so x y = x (c v + + c k v k ) = c ( x v ) + + c k ( x v k ) = c () + + c k () =. Thus x S and { v... v k } S. This shows S = { v... v k }. We will use Proposition 6.2.4 shortly to describe how to find the orthogonal complement of a subspace more systematically. Recall that range(a) is the span of the columns of A. It now becomes of use to us to also look at the span of the rows of a matrix. Definition 6.2.5. Let row(a) denote the span of the rows of matrix A. For example ([ ]) 2 row = span 2. Let the transpose A of a matrix A denote A with its rows and columns interchanged. For example [ ] 2 = 2 Notice that range(a) = row(a ) and that (A ) = A. Now we can formalize how to find the orthogonal complement of a subspace. Theorem 6.2.6. For any matrix A row(a) = ker(a) and range(a) = ker(a ). Proof. Suppose A is an m n matrix with rows v... v m from top to bottom. Then row(a) = span( v... v m ) and () row(a) = (span( v... v m )) = { v... v m } = { x R n x v =... x v m = }. Now notice that x v =... x v m = are exactly the same equations we solve to find ker(a). For example if v = 2 then x x v = 2 = x + 2 + x = x which is the equation the row [ 2 ] gives when finding ker(a). Thus so row(a) = ker(a) by (). Finally notice that ker(a) = { x R n x v =... x v m = } range(a) = row(a ) = ker(a ).
Example 6.2.7. Find bases for the following orthogonal complements. (a) span 2 4 (b) span 2 Solutions: (a) span 2 4 (b). 2 4 = row ([ 2 4 ]) ([ ]) = ker 2 4 = span. span 2 = row = ker ([ ]) ([ ]) = ker 2 2 ([ ]) 4 4 = span 2 2. In fact as you may have already guessed from the name S is in fact a subspace complementary to S. As opposed ([ to just([ any complementary ([ subspace the orthogonal complement ([ of S is unique. For example ([ 5 2 span span span are all subspaces complementary of span but only span ]) ]) ]) ]) ]) ([ is the orthogonal complement of span. The reader may verify that the only subspace complementary ]) to S that is also orthogonal to S is the orthogonal complement of S Problem 4. After proving that S S are complementary subspaces we mention other properties that orthogonal complements have. S S Theorem 6.2.8. For any subspace S R n S and S are complementary subspaces in R n. Proof. First we show that S S = { }. Suppose x S S. Then since x S and x S x must be orthogonal to itself: x x = = x =
4 by Proposition 6..2(iii). Therefore S S = { }. Next we show that S + S = R n using dimension. Let ( v... v k ) be a basis for S so that dim(s) = k. Then S = (span( v... v k )) = { v... v k } = { x R n x v =... x v k = }. So S is the set of solutions to a homogeneous system of k equations in n variables which means there are at most k leading variables and so at least n k free variables. Hence dim(s ) n k. Then dim(s + S ) = dim(s) + dim(s ) dim(s S ) = k + dim(s ) k + (n k) = n. But S + S R n so dim(s + S ) n as well. This forces dim(s + S ) = n which forces S + S = R n so S and S are complementary subspaces of R n by Definition.5.2. Proposition 6.2.9. For any subspaces S S S 2 R n (a) dim(s ) = n dim(s). (b) If S S 2 then S2 S (c) ((S) ) = S (d) (S + S 2 ) = S S2 (e) (S S 2 ) = S + S2. Proof. Problem 5. Now that we know S and S are complementary subspaces we can discuss the orthogonal projection π SS onto S with respect to S which we abbreviate π S. Definition 6.2.. Suppose S R n is a subspace. Then is the projection onto S with respect to S. Using Proposition 6.2.9 (c) we also have Thus for all x R n (2) π S = π SS π S = π S (S ) = π S S. x = π S ( x) + π S ( x) In order to describe a formula for the orthogonal projection we will need the following Theorem to be proven in Section 6.. Theorem 6.2.. Every subspace of R n has an orthogonal basis. Proof. See Theorem 6... But that doesn t mean you can t try it yourself first. Proposition 6.2.2. Suppose S R n is a subspace with orthogonal basis ( u... u k ). Then x u x uk π S ( x) = u 2 u + + u k 2 u k Proof. Let ( v... v n k ) be an orthogonal basis for S which exists by 6.2.. Then ( u... u k v... v n k ) is an orthogonal basis for R n Problem. This means for any x R n x span( u... u k v... v n k ). Hence by Proposition 6..2 x = (( x u u 2 ) u + + ( ) ) x uk u k 2 u k + ( ) x v x vn k v 2 v + + v n k 2 v n k.
5 Since we can conclude that x u x uk u 2 u + + u k 2 u k S x v x vn k v 2 v + + v n k 2 v n k S x u x uk π S ( x) = π SS ( x) = u 2 u + + u k 2 u k x v x vn k π S ( x) = π S S( x) = v 2 v + + v n k 2 v n k. Exercises:. Find bases for the following orthogonal complements. 2 (a) span 4 7 4 (b) span 2 2 2 (c) span 5 7 6 (d) span 5 4 (e) span. 4 2 2. Find formulas for π S ( x) for each of the following subspaces S. (a) S = span 2 (b) S = span 2 (c) S = span 2 2 6 (d) S = span 4 5 2
6 (e) S = span 2. Problems:. (2) Suppose S R n is a subspace ( u... u k ) is an orthogonal basis for S and ( v... v n k ) is an orthogonal basis for S. Show that ( u... u k v... v n k ) is orthogonal basis for R n. 2. (2) Suppose ( u... u k v... v n k ) is orthogonal basis for R n. Show that span( u... u k ) = span( v... v n k ).. (2+) Fix x R n and a subspace S R n. Show that π S ( x) is the unique vector in R n so that (i) π S ( x) S (ii) x π S ( x) S. 4. (2+) Suppose S R n is a subspace. Show that for all x y R n 5. () Prove Proposition 6.2.9. x y = π S ( x) π S ( y) + π S ( x) π S ( y) 6. (a) (2) Show that if A C (A and C differ by EROs) then row(a) = row(c). (b) () Show that the nonzero rows in RREF(A) form a basis for row(a). 7. () Prove or give a counterexample: For any matrix A rank(a) = rank(a ). 8. () Prove or give a counterexample: For any matrix A nullity(a) = nullity(a ). 9. (2) Show that if the rows of m n matrix A are linearly independent then rank(a) = m and nullity(a) = n m.. ) Show that for any n n matrix A det(a) = det(a ).. () Suppose ( u... u n ) is a basis for R n and b... b n are scalars. Show that there exists a unique x R n so that x u = b... x u n = b n. 2. (+) Suppose a... a n R m and b... b n are scalars. Let Show that there exists x R n so that if and only if ker(a) ker(b). A = [ a... a n ] B = [ b... b n ]. x u = b... x u n = b n.. () Suppose ( u... u k ) is orthonormal in R n and let S = span( u... u k ). Then for x R n show that x S if and only if x 2 = ( x u ) 2 + + ( x u k ) 2. 4. () Suppose S S 2 R n are complementary subspaces so that Show that S 2 = S. 5. Suppose S S 2 R n are subspaces with S S 2. x y = for all x S y S 2.
7 (a) () Show that for all y S π S2 ( y) S. (b) () Show that for each x R n there exists unique u S v S 2 S w S2 x = u + v + w. (c) () Show that π S π S2 = π S. so that 6. (4) Fix x R n and a subspace S R n. Show that the minimum of x y 2 over all y S is acheived uniquely when y = π S ( x). 7. (4) Suppose P : R n R n is a projection map which satisfies Show that P is an orthogonal projection. P ( x) x for all x R n.