Orthogonal Complements Definition Let W be a subspace of R n. If a vector z is orthogonal to every vector in W, then z is said to be orthogonal to W. The set of all such vectors z is called the orthogonal complement of W, denoted W. W is a subspace of R n (HW8 - Problem 2) z W z is orthogonal to all the vectors in any basis of W Let A be an m n matrix, then [Row(A)] = Nul(A) and [Col(A)] = Nul(A T ) Quang T. Bach Math 18 December 1, 2017 1 / 13
Orthogonal Sets Definition A set of vectors S = {v 1, v 2,..., v p } in R n is said to be an orthogonal set if every pair of distinct vectors in S is orthogonal. Namely, v i v j = 0 whenever i j In addition, if each vector in S is a unit vector then S is called orthonormal. (Orthogonality Implies Linearly Independence) If S = {v 1, v 2,..., v p } is an orthogonal set of non-zero vectors in R n, then S is linearly independent and thus forms a basis for the subspace spanned by S. Quang T. Bach Math 18 December 1, 2017 2 / 13
Question 1 Which of the following set is not orthogonal? { [ ] [ ] 1 2 } A., 2 1 { [ 1/ ] [ ] 5 B. 2/ 2/ 5 }, 5 1/ 5 { [ ] [ ] [ ] 1 2 0 } C.,, 2 1 0 { [ ] [ ] [ ] 1 2 2 } D.,, 2 1 4 E. (C) and (D) Quang T. Bach Math 18 December 1, 2017 3 / 13
Question 2 What is the maximum size of a non-zero orthogonal set in R 3? A. 2 B. 3 C. 4 D. More than 4 but the number is finite E. Infinity many Quang T. Bach Math 18 December 1, 2017 4 / 13
Orthogonal Matrices An m n matrix U is called orthogonal if its columns form a set of orthonormal vectors. Remark: Orthogonal matrix does not have to be square. If U is an m n orthogonal matrix then U T U = I n If U is square and is an orthogonal matrix, then U 1 = U T (HW7 - Problem 3) Question 3. Let A be an m n orthogonal matrix. Which of the following must be false about its dimensions m and n? A. m = n B. m > n C. m < n D. Choose this if all the above are true Quang T. Bach Math 18 December 1, 2017 5 / 13
Linear Transformation using Orthogonal Matrix Let U be an m n matrix with orthonormal columns and let x and y be vectors in R n. Then a. Ux = x b. (Ux) (Uy) = x y c. (Ux) (Uy) = 0 if and only if x y = 0 Note: The above theorem shows that the linear transformation x Ux preserves lengths and orthogonality. (HW8 - Problem 1) Quang T. Bach Math 18 December 1, 2017 6 / 13
Orthogonal Basis Definition If an orthogonal (orthonormal) set S is also a basis for a vector space V then S is called an orthogonal basis (orthonormal basis) for V. Example The following sets are orthonormal bases in R 3 : [ [ 1 1 T 2 2 S 1 = { 2,, 0], 2 6, 6, 2 ] T [ 2 2, 3 3, 2 3, 1 ] T } 3 S 2 = {[cos(θ), sin(θ), 0] T, [ sin(θ), cos(θ), 0] T, [0, 0, 1] T } To see this, check that each set is an orthonormal set of linearly independent vectors in R 3. Quang T. Bach Math 18 December 1, 2017 7 / 13
Orthogonal Projection Given a vector y R n and a non-zero vector u R n. We can always decompose y into a sum of two vectors y = ŷ + z where ŷ is a multiple of u and z is orthogonal to u. The vector ŷ is called the orthogonal projection of y onto u and the vector z is the component of y orthogonal to u. In general, the projection of y onto Span{u} is given by ŷ = y u u u u We sometimes denote this by proj L (y) where L is the subspace spanned by u. Quang T. Bach Math 18 December 1, 2017 8 / 13
Orthogonal Projection - Example Example Find the orthogonal decomposition of y = decompose y. [ ] 7 onto u = 6 [ ] 4 and 2 Answer: ŷ = y u u u [ ] 28 + 12 4 u = 16 + 4 u = 2 2 [ ] 1 z = y ŷ =. 2 = [ ] 8. 4 Quang T. Bach Math 18 December 1, 2017 9 / 13
Coordinates under Orthogonal Basis Let S = {v 1, v 2,..., v p } be an orthogonal basis for a subspace W of R n. For each u W, the weights in the linear combination u = c 1 v 1 + c 2 v 2 + + c p v p is given by c j = u v j v j v j Quang T. Bach Math 18 December 1, 2017 10 / 13
Orthogonal Decomposition Let W be a subspace of R n with an orthogonal basis S = {v 1, v 2,..., v p }. Then every vector y in R n can also be written as y = ŷ + z where and ŷ = y v 1 v 1 v 1 v 1 + y v 2 v 2 v 2 v 2 + + y v p v p v p v p W z = y ŷ W We sometimes denote ŷ, the orthogonal projection of y onto W, by proj W (y) Quang T. Bach Math 18 December 1, 2017 11 / 13
Orthogonal Decomposition - Example Example 1 { 2 2 } Let y = 2 and W = Span{v 1, v 2 } = 5, 1. Decompose y 3 1 1 into the sum of a vector in W and a vector in W. ŷ = y v 1 v 1 v 1 v 1 + y v 2 v 2 v 2 v 2 = 3 10 Check that ŷ z 2 5 + 1 2 2/5 1 = 2 W 2 1 1 1/5 1 2/5 7/5 z = y ŷ = 2 2 = 0 W 3 1/5 14/5 Quang T. Bach Math 18 December 1, 2017 12 / 13
Orthogonal Projection - Properties Lemma If y W then proj W (y) = y Let W be a subspace of R n. Let y be any vector in R n and let ŷ be the orthogonal projection of y onto W. Then ŷ is the closest point in W to y, in the sense that for all v W and v ŷ d(y, ŷ) = y ŷ < y v = d(y, v) The vector ŷ is called the best approximation to y by elements of W. Quang T. Bach Math 18 December 1, 2017 13 / 13