Lecture 0: Vector Algebra: Orthogonal Basis Orthogonal Basis of a subspace Computing an orthogonal basis for a subspace using Gram-Schmidt Orthogonalization Process
Orthogonal Set Any set of vectors that are mutually orthogonal, is a an orthogonal set.
Orthonormal Set Any set of unit vectors that are mutually orthogonal, is a an orthonormal set. In other words, any orthogonal set is an orthonormal set if all the vectors in the set are unit vectors. Example: u, u 2, u 3 is an orthonormal set, where, 3 u =, u 2 = 6 2 6, u 3 = 66 4 66 7 6 66
An orthogonal set is Linearly Independent
Projection of vector b on vector a a b = a b cos θ Vector c is the image/perpendicular projection of b on a Direction of c is the same as a Magnitude of c is c = b cos θ = a b a If a is the unit vector of a, then vector c = a b a a b a = a a a c = a b = a b a a a b cos θ b b sin θ
Orthogonal Basis An orthogonal basis for a subspace W of R n is a basis for W that is also an orthogonal set. Example: 0 0, 0 0, 0 0 is basically the x, y, and z axis. It is an orthogonal basis in R 3, and it spans the whole R 3 space. It is also an orthogonal set.
Orthogonal Basis We know that given a basis of a subspace, any vector in that subspace will be a linear combination of the basis vectors. For example, if u and v are linearly independent and form the basis for a subspace S, then any vector y in S can be expressed as: y = c u + c 2 v But computing c and c 2 is not straight forward. v y On the other hand, if u and v form an orthogonal basis, then c = y u u y u = u u and c 2 = y v v y v = v v c 2 c u
Does not work if it is not Orthogonal basis y = c u + c 2 v But computing c and c 2 is not straight forward (yet). What is computed is, d = y u u = y u u u d 2 v y d 2 = y v v = y v v v c 2 c u d 8
Orthogonal Decomposition Theorem
Orthogonal Decomposition Theorem
Orthogonal Decomposition Theorem
Orthogonal Decomposition Theorem Example
Projection of a vector on a Subspace u and v are orthogonal 3D vectors. They span a plane (green plane) in 3D y is an arbitrary 3D vector out of the plane. y is the projection of y onto the plane. y = y u y v u + v u u v v The point y is also the closest point to y on the plane. y y is perpendicular to y, Span{u, v}, and hence u and v v u y y Perpendicular to y, u, v
Closest point of a vector to a span does not depend on the basis of that span y is an arbitrary 3D vector out of the plane. y is the projection of y onto the plane. y = x u + x 2 u 2 y = x 3 u 3 + x 4 u 4 y = x 5 u + x 6 u 3 u 2 u Coordinates of y change if the basis changes. But the vector y itself does not change. Hence the the closest point to y on the plane does not change. Even here, y y is perpendicular to y and Span{.,. } u 4 u 3 y y Perpendicular to y and the span
Closest point of a vector to a span does not depend on the basis of that span FINDING THE CLOSEST POINT OF A VECTOR TO A SPAN means : Finding the coordinates of the projection of the vector So, if you want to compute the closest point of a vector to a span, then find an appropriate basis with which you can compute the coordinates of the projection easily. What would be that basis? Answer: An orthogonal basis! u 4 u 3 y y u 2 u Perpendicular to y and the span
Projection on a span of non-orthogonal vectors How to find projection of any arbitrary 3D vector onto the span of two non-orthogonal, linearly independent vectors? u and u 2 are not orthogonal, but linearly independent vectors in 3D. y is an arbitrary 3D vector. Find the projection of y in the space spanned by u and u 2. a) First, find the orthogonal set of vectors v and v 2 that span the same subspace as u and u 2. In other words, find an orthogonal basis. b) Project y onto the space spanned by orthogonal v and v 2 vectors, as we earlier. v 2 u 2 u v y y
How to find an orthogonal basis?
How to find an orthogonal basis? Assume that the first vector u is in the orthogonal basis. Other vector(s) of the basis are computed that are perpendicular to u Let v = u Let v 2 = u 2 u 2v v v v We know that v 2 is perpendicular to v. v 2 is in the Span{u, u 2 } (Why?) So Span{u, u 2 } = Span{v, v 2 } And {v, v 2 } is an orthogonal basis Projection of y on to the Span{u, u 2 } v 2 u 2 u v y y y = y v v v v + y v 2 v 2 v 2 v 2
Gram-Schmidt Orthogonalization Process
Gram-Schmidt Orthogonalization Process Example:
Solving Inconsistent Systems
Solving Inconsistent Systems
Solving Inconsistent Systems Example : A trader buys and/or sells tomatoes and potatoes. (Negative number means buys, positive number means sells.) In the process, he either makes profit (positive number) or loss (negative number). A week s transaction is shown; find the approximate cost of tomatoes and potatoes. Tomatoes (tons) Potatoes (tons) -6 - -2 2 7 6 Profit/Loss (in thousands)
Solving Inconsistent Systems t - 6p = t - 2p = 2 t + p = t + 7p = 6 t + 6 2 7 p = 2 6 The above equation might not have a solution (values of t and p that would satisfy that equation). So the best we can do is to find the values of t and p that would result in a vector on the right hand side that is as close as possible to the desired right hand side vector.
Solving Inconsistent Systems Example 2: