Math 6 Lecture Notes: Sections 6., 6., 6. and 6. Orthogonal Sets and Projections We will not cover general inner product spaces. We will, however, focus on a particular inner product space the inner product space of R n with the dot product. Notation. When discussing vectors in an inner product space and not strictly dot products, a slightly different notation is used. The inner product we primarily use will be the dot product; however, a general inner product is denoted by u, v. Thus when dealing with dot products, u v = u, v. Like vector spaces, we can define this operation differently; most of the time, however, we won t. Definition. Two vectors are orthogonal if their dot product is zero. That is, vectors u and v are orthogonal if u v = 0. Note that for two-dimensional and three-dimensional vectors this means the angle between the two vectors is 90. A set of vectors { v, v,..., v n } is orthogonal if the vectors are mutually orthogonal; that is, for every i j, v i v j = 0. A vector is normal it has a length of unit; that is, if v i = for i =... n. A set of vectors { v, v,..., v n } is orthonormal if the vectors are mutually orthogonal and each vector is normal. Example. Show that the basis B for R below is an orthogonal set. Construct an orthonormal basis from this set. {[ ] [ ]} B =, Figure 5 5 5 5 y x
Example. Show that the set B is orthogonal. If this set is a basis for R, construct an orthormal basis from it. 0 B = 0,, 0 0 Instructor: A.E.Cary Page of 9
Recall from Calculus III that a scalar projection of u onto v is given by comp v u = projection of u onto v (or the orthogonal projection of u onto v ) is given by proj v u = We can also write comp v u = u v v and proj v u = u v v v v u v v and the vector u v v v v. Figure 5 y x 5 5 5 Figure 5 y x 5 5 5 Example. Derive the above formulas for comp v u and proj v u. Instructor: A.E.Cary Page of 9
Example. What can we say about proj v u and u proj v u? The two are orthogonal! Verify this. Example 5. Use the standard inner product on R (the dot product) and find proj v u and u proj v u for u = and v =. Confirm that they are orthogonal. Figure 5 y x 5 5 5 Instructor: A.E.Cary Page of 9
Orthogonal Decomposition and the Gram-Schmidt Process Definition. Let W be a subspace of R n. The set of all vectors z that are orthogonal to the vectors in W is called the orthogonal complement of W and is denoted by W. Theorem. A vector x is in W if and only if x is orthogonal to every vector in a set that spans W. Furthermore, W is itself a subspace of R n. Theorem. The Orthogonal Decomposition Theorem Let W be a subspace of R n. Then each y in R n can be written uniquely in the form y = ŷ + z where ŷ is in W and z is in W. Furthermore, if { u,..., u p } is an orthogonal basis for W, then and z = y ŷ. ŷ = y u u u u + + y u p u p u p u p Definition. Let W be a subspace of R n and let { u,..., u p } be an orthogonal basis for W. The orthogonal projection of y onto W, denoted by ŷ or proj W y is ŷ = y u u u u + + y u p u p u p u p Instructor: A.E.Cary Page 5 of 9
Example 6. Verify that { u, u } is an orthogonal set and then find the orthogonal projection of y onto Span { u, u }. u = u = y = Theorem. The Best Approximation Theorem Let W be a subspace of R n, let y be any vector in R n, and let ŷ be the orthogonal projection of y onto W. Then ŷ can be used to determine the closest point in W to y as for all v in W distinct from ŷ. y ŷ < y v Example 7. State the closest point in W to the vector y for the previous example. Instructor: A.E.Cary Page 6 of 9
Theorem. Every finite dimensional inner product space has an orthogonal basis. Gram-Schmidt Process This process is an algorithm for finding an orthonormal basis to any inner product space, and can be used to find an orthonormal basis to any vector space. Gram-Schmidt Process. () Let B = { v, v,..., v n } be any basis for the inner product space V. () Use B to define a set of n vectors { w, w,..., w n } as follows: w = v w = v proj w v w = v proj w v proj w v w = v proj w v proj w v proj w v. w n = v n proj w v n proj w v n proj wn v n () The set B = { w, w,..., w n } is an orthogonal basis for V. () To obtain an orthonormal basis, we divide each vector in B by its length. The basis B below is an orthonormal basis for V : { } B w = w, w w,..., w n w n Example 8. Use the standard inner product on R, the basis B = 0, 0, and the Gram-Schmidt process to find an orthonormal basis for R. Instructor: A.E.Cary Page 7 of 9
Instructor: A.E.Cary Page 8 of 9
Example 9. Find an orthogonal basis for the column space of the matrix A below. Use this to then find an orthonormal basis for the column space of A. 6 6 8 6 Instructor: A.E.Cary Page 9 of 9