6. Orthogonal Projections Math 2 Linear Algebra 6. Orthogonal Projections Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/ jiwenhe/math2 Jiwen He, University of Houston Math 2, Linear Algebra / 6
6. Orthogonal Projections Orthogonal Projection: Review Orthogonal Projection: Examples The Orthogonal Decomposition Theorem The Orthogonal Decomposition: Example Geometric Interpretation of Orthogonal Projections The Best Approximation Theorem The Best Approximation Theorem: Example New View of Matrix Multiplication Orthogonal Projection: Theorem Jiwen He, University of Houston Math 2, Linear Algebra 2 / 6
Orthogonal Projection: Review ŷ = y u u uu is the orthogonal projection of onto. Suppose {u,..., u p } is an orthogonal basis for W in R n. each y in W, ( ) ( ) y u y up y = u + + u p u u u p u p For Jiwen He, University of Houston Math 2, Linear Algebra / 6
Orthogonal Projection: Example Example Suppose {u, u 2, u } is an orthogonal basis for R and let W =Span{u, u 2 }. Write y in R as the sum of a vector ŷ in W and a vector z in W. Jiwen He, University of Houston Math 2, Linear Algebra 4 / 6
Orthogonal Projection: Example (cont.) Solution: Write ( ) ( ) ( ) y u y u2 y u y = u + u 2 + u u u u 2 u 2 u u where ( ) ( ) ( ) y u y u2 y u ŷ= u + u 2, z = u. u u u 2 u 2 u u To show that z is orthogonal to every vector in W, show that z is orthogonal to the vectors in {u, u 2 }. Since z u = = = z u 2 = = = Jiwen He, University of Houston Math 2, Linear Algebra 5 / 6
The Orthogonal Decomposition Theorem Theorem (8) Let W be a subspace of R n. Then each y in R n can be uniquely represented in the form y =ŷ + z where ŷ is in W and z is in W. In fact, if {u,..., u p } is any orthogonal basis of W, then ( ) ( ) y u y up ŷ = u + + u p u u u p u p and z = y ŷ. The vector ŷ is called the orthogonal projection of y onto W. Jiwen He, University of Houston Math 2, Linear Algebra 6 / 6
The Orthogonal Decomposition Theorem Jiwen He, University of Houston Math 2, Linear Algebra 7 / 6
The Orthogonal Decomposition: Example Example Let u =, u 2 =, and y =. Observe that {u, u 2 } is an orthogonal basis for W =Span{u, u 2 }. Write y as the sum of a vector in W and a vector orthogonal to W. Solution: ( ) proj W y = ŷ = y u u u u + + ( ) = ( ) z = y ŷ = ( ) y u2 u 2 u 2 u 2 = = 9 Jiwen He, University of Houston Math 2, Linear Algebra 8 / 6
Geometric Interpretation of Orthogonal Projections Jiwen He, University of Houston Math 2, Linear Algebra 9 / 6
The Best Approximation Theorem Theorem (9 The Best Approximation Theorem) Let W be a subspace of R n, y any vector in R n, and ŷ the orthogonal projection of y onto W. Then ŷ is the point in W closest to y, in the sense that for all v in W distinct from ŷ. y ŷ < y v Jiwen He, University of Houston Math 2, Linear Algebra / 6
The Best Approximation Theorem (cont.) Outline of Proof: Let v in W distinct from ŷ. Then v ŷ is also in W (why?) z = y ŷ is orthogonal to W y ŷ is orthogonal to v ŷ y v = (y ŷ) + (ŷ v) = y v 2 = y ŷ 2 + ŷ v 2. y v 2 > y ŷ 2 Hence, y ŷ < y v. Jiwen He, University of Houston Math 2, Linear Algebra / 6
The Best Approximation Theorem: Example Example Find the closest point to y in Span{u, u 2 } where 2 y = 4, u =, and u 2=. 2 Solution: ( ) ( ) y u y u2 ŷ= u + u 2 u u u 2 u 2 = ( ) + ( ) = Jiwen He, University of Houston Math 2, Linear Algebra 2 / 6
New View of Matrix Multiplication Part of Theorem below is based upon another way to view matrix multiplication where A is m p and B is p n AB = [ col A col 2 A col p A ] row B row 2 B. row p B = (col A) (row B) + + (col p A) (row p B) Jiwen He, University of Houston Math 2, Linear Algebra / 6
New View of Matrix Multiplication: Example Example = [ 5 6 [ 5 ] [ 2 4 2 [ 5 6 ] = ] [ 2 4 2 ] [ 2 ] + [ 6 [ 4 5 7 ] ] ] [ 4 2 ] = Jiwen He, University of Houston Math 2, Linear Algebra 4 / 6
Orthogonal Matrix So if U = [ ] u u 2 u p. Then U T u T = 2. u T u T p. So UU T = u u T + u 2u T 2 + + u pu T p ( UU T ) y = ( u u T + u 2u T 2 + + u pu T ) p y = ( u u T ) ( y + u2 u T ) ( 2 y + + up u T ) p y ( = u u T y ) ( + u 2 u T 2 y ) ( + + u p u T p y ) = (y u ) u + (y u 2 ) u 2 + + (y u p ) u p ( UU T ) y = (y u ) u + (y u 2 ) u 2 + + (y u p ) u p Jiwen He, University of Houston Math 2, Linear Algebra 5 / 6
Orthogonal Projection: Theorem Theorem () If {u,..., u p } is an orthonormal basis for a subspace W of R n, then proj W y = (y u ) u + + ( y u p ) up If U = [ u u 2 u p ], then Outline of Proof: proj W y =UU T y for all y in R n. proj W y = ( y u u u ) u + + ( y up u p u p ) u p = (y u ) u + + ( y u p ) up = UU T y. Jiwen He, University of Houston Math 2, Linear Algebra 6 / 6