The Gram-Schmidt Process In this section all vector spaces will be subspaces of some R m. Definition.. Let S = {v...v n } R m. The set S is said to be orthogonal if v v j = whenever i j. If in addition v i = for each i then we say S is orthonormal. The goal of this section is to answer the following question. Given a basis for a vector space V how can we find an orthonormal basis for V? First we verify that an orthogonal set is linearly independent. Theorem.. Let {v...v n } R m be a set of nonzero vectors and suppose v v j = whenever i j. Then {v...v n } is linearly independent. Proof for n=4. Suppose c v + c v + c 3 v 3 + c 4 v 4 =. ) We shall solve for c by taking the dot product of both sides with v. v c v + c v + c 3 v 3 + c 4 v 4 ) = v c v v + c v v + c 3 v v 3 + c 4 v v 4 ) = c v + c + c 3 + c 4 = c v = Since v we have that c =. We shall solve for c by taking the dot product of both sides of ) with v. v c v + c v + c 3 v 3 + c 4 v 4 ) = v c v v + c v v + c 3 v v 3 + c 4 v v 4 ) = c + c v + c 3 + c 4 = c v = Since v we have that c =. Likewise c 3 = and c 4 =. Thus {v v v 3 v 4 } is linearly independent. c Michael C. Sullivan July 7 8
Problem. Write out the general proof for Theorem.. Theorem.3 The Gram-Schmidt Process). Let {u...u n } be a basis for a vector space V. Let v = u. For k =...n let v k = u k k i= v i u k v i v i v i and for k =...n let w i = v i v i. Then {v...v n } is an orthogonal basis for V and {w...w n } is an orthonormal basis for V. Explanation. Let s suppose n = 4 and write out the formulas. Of course v = u is straight forward. Recall from?? that the projection of u in the direction of v is proj v u = v u v v v. Thus in the formula below we project u in the direction of v and subtract this from u. What is left will be perpendicular to v. v = u v u v v v Next we project u 3 first in the direction of v and then in the direction of v. Subtracting these from u 3 produces a vector perpendicular to both v and v. v 3 = u 3 v u 3 v v u 3 v v v v v Finally we find the projections of u 4 in the directions of v v and v 3. Subtracting these from u 4 produces a vector perpendicular to v v and v 3. v 4 = u 4 v u 4 v v v v u 4 v v v v 3 u 4 v 3 v 3 v 3 Proof of Theorem.3. We check that v is perpendicular to v. v v = v u v ) u v = v u v u v v = v u v u = v v v v Next we check that v 3 is perpendicular to v. v v 3 = v u 3 v u 3 v v ) u 3 v = v v v v
v u 3 v u 3 v v v v v u 3 v v v v = v u 3 v u 3 = We used the fact that we already knew v v =. Now we check that v 3 is perpendicular to v. v v 3 = v u 3 v u 3 v v ) u 3 v = v u 3 v u 3 =. v v v v Again we used v v =. We can continue like this. Next we would show that v 4 is perpendicular to v v and v 3. The reader should be able to work this out and see that the calculations are similar to what we have just done. We use the Principle of Mathematical Induction to cover all positive integers n. Assume we know that v k is perpendicular to v...v k. Let v j {v...v k }. We will show v k v j =. v j v k = v j u k k i= v i u k v i v i v i ) = v j u k v j u k v j u k = k i= v i u k v i v i v j v i = since v j v i = unless i = j. Therefore {v...v n } is orthogonal. There are no zero vectors in it since the original vectors {u...u n } were linearly independent. Dividing each vector by its magnitude produces an orthonormal set. {[ Example [. ]} Find an orthonormal basis for the vector space spanned by 8. Answer. We don t need the Gram-Schmidt Process. These ] 4 vectors span R. We can just use the standard basis for R. Example. Find an orthonormal basis for the vector space spanned by. 3
Solution. Let v =. Then v = u v u v v v = u v = Next v 3 = u 3 v u 3 v v v v u 3 v v v = u 3 v 4 v = 3/ 3/ Dividing each of these by its length gives / / / / / / / / Problem. Find an orthonormal basis for each vector space with the given basis below. {[ ]} a. b. 3 c. 3 d. e. 3 3 f. 3 Problem 3. Find an orthonormal basis for the plane given by x+3y z =. 4
Problem 4. Consider w 3 x y = 3 4 8 z. a. Find the basis for the solution space produced by putting the matrix into reduced row echelon form. b. Find an orthonormal basis for the solution space. c. Find the transition matrix that will convert coordinates vectors with respect to the first basis into convert coordinates vectors with respect your orthonormal basis. Problem 5. Find orthonormal bases for the eigenspaces of 4. 4. Maple Command for the Gram-Schmidt Process Here is an example illustrating how to use Maple s GramSchmidt command. The command is part of the LinearAlgebra package. If you leave off the normalized option the GramSchmidt command will return an orthogonal set of vectors that have not been normalized. > GramSchmidt[<><><33>]normalized); /3 3 /3 3 /3 3 /5 5 /5 5 /5 5 /5 5 /5 / /5 / 5
. Inner Product Spaces [Optional] Definition.4. An Inner product space is a vector space V together with a function taking pairs of vectors to the reals with the properties listed below. The function is called an inner product and is denoted by vu. Let r R and let u v and w be in V. vv with equality holding only when v =. vu = uv. rvu = r vu. u + vw = uw + vw. The dot product in any subspace of R n is an example. Here is another. Let P be the set of all polynomials regarded as a vector space. For f and g in P define f g = fx)gx) dx. For example x + x + x = x + )x + x) dx = x3 + x + xdx = 4/3. The reader should check that this gives an inner product space. The results about projections orthogonality and the Gram-Schmidt Process carry over to inner product spaces. The magnitude of a vector v is defined as vv. Problem 6. In an inner product space prove that v ru = r vu. Problem 7. In an inner product space prove that v =. Problem 8. Let S = span {sinx) sinx) sin3x) sin4x)}. Let the inner product be f g = π fx)gx) dx. Show that S is orthogonal and hence forms a basis for S. Find an orthonormal basis for S. Hint: sinm + n)x) sinm n)x) sin mx sin nxdx = + + C m + n) m n) for m ±n. But you knew that! Problem 9. For the inner product used above for polynomials compute x x 3 x + x 5 + x 4 and x + 3x + x 3 x. 6
Problem. a. Find an orthonormal basis for P 4 with inner product above starting with the standard basis { x x x 3 x 4 }. b. Find the transition matrix that takes coordinate vectors with respect to the standard basis to coordinate vectors with respect to the basis you found. [ ] Problem. Let A =. For v and u in R 3 define v u = v T Au. [ ] [ ] a. Compute. [ ] [ 3 ] 5 b. Compute. c. Prove that is an inner product on R. 7