. Length, Angle, and Orthogonality In this section, we discuss the defintion of length and angle for vectors. We also define what it means for two vectors to be orthogonal. Then we see that linear systems are much easier to solve if their set of vectors is orthogonal. We begin by defining the dot product, which has already shown up on defining matrix-vector and matrix-matrix multiplication. It is essential to be able to define length and angle. Definition.. The dot product of two vectors x, y R n is given by x x y =.. y. = y + x y x + + x n y n y n x n y For example, 3 In addition, the dot product has the following properties. Proposition.. Given x, y, z R n and a scalar c, (i) x y = y x, (ii) x x 0 (iii) x x = 0 if and only if x = 0, (iv) x (c y) = (c x) y = c( x y), (v) ( x + y) z = x z + y z, 7 = () + 3(7) () =. Proof. Problem. With the dot product defined, we can now define length of vectors and angle between two vectors. Definition.3. The length of a vector x R n is given by x = x x. The angle between two nonzero vectors x, y R n is given by ( ) x y ( x, y) = cos. x y Wait, we already have notions of length and angle from geometry. Let us check that our definitiona agree with our previous notions. First, the definition of length can be rewritten as x = x + x + + x n. Clearly, for x R, x = x = is consistent with our geometric notion of length in R. Using the Pythagorean Theorem, we can see this is consistent with our geometric notion of length in R :
x + x x Iterating the Pythagorean theorem, one can show that this length is consistent with our geometric notion of length in R n. We will often use the form x = x x. We also take this opportunity to note some of the properties of the length function. Proposition.. Given any x, y R n and scalar c, (i) x 0. (ii) x = 0 if and only if x = 0 (iii) c x = c x (iv) x + y + x y = ( x + y ), called the Parallelogram Law. We often restrict to unit length. Definition.. A vector x R n is a unit vector if x =. The set of unit vectors in R n can be thought of as the set of directions in R n since magnitude is taked out of the equation by restricting to unit length. As for angles, consider nonzero x, y R n. Then, x, y and y x form a triangle in span{ x, y}. Usually, span{ x, y} will be a -dimensional plane, and if not it is a line. If it is a line, embed it in a some -dimensional plane. Then, we have y x x ( x, y) y Now, by the Law of Cosines, Using (??) and some properties of the dot product, y x = y + x x y cos( ( x, y)). ( y x) ( y x) = y y + x x x y cos( ( x, y)) y y ( x y) + x x = y y + x x x y cos( ( x, y)) ( x y) = x y cos( ( x, y)) cos( ( x, y)) = x y x y ( x, y) = cos ( x y x y ).
3 Thus, the definition of angle agrees with our geometrid definition of angle. We are particularly interested in when nonzero two vectors are orthogonal/perpendicular, which is when their angle is π. Notice ( x, y) = π ( ) x y cos = π x y = 0 x y = 0. x y x y This means the following definition agrees with our geometric notions. Definition.6. Two vectors x, y R n are orthogonal, denoted x y, if x y = 0. We extend this definition to finite sets of vectors by requiring that each pair of vectors in that set be othogonal. Definition.7. A set { v,..., v k } R n is orthogonal if each vector is nonzero, and v i v j = 0 for all i j. Also, { v,..., v k } R n set is orthonormal if in addition all of the vectors are unit vectors, so { if i = j v i v j = 0 if i j. The canonical example of an orthonormal set is the set of standard basis vectors in R n, { e,..., e n }. We have { if i = j e i e j = 0 if i j. because in order for their dot product to be nonzero, the two ones need to be in the same component, in which case their dot product is. Here are some other examples of orthogonal sets of vectors: 0 0 0,, 3,,,, 0 3 3, 3, 3 3, 0 Even one pair of vectors not being orthogonal forces the whole set to not be orthogonal. For example, 3 0, 3,, 0 3 is not orthogonal because 0 and 0 are not orthogonal, but every other pair is. Although 0 is orthogonal to every vector, we exclude 0 from orthogonal sets for reasons that will be clear later. Why do we care about orthogonal sets of vectors? They have applications to physics, differential equations, and other areas of advanced matematics. But for this class, they make linear systems easy to solve, and they will play an important role in constructing orthogonal linear maps (Section ). If a linear system uses an orthogonal set of vectors, we can simply our computations by a lot. Proposition.8. Suppose { v,..., v k } R n is orthogonal, and x R n. Then, c v + + c k v k = x has at most one solution. If it has a solution, it is given by Proof. Suppose that c,..., c k satisfy c j = x v j, for all j. v j c v + + c k v k = x
In order to use that { v,..., v k } is orthogonal, we have to get the dot products v i v j involved somehow. We can do that simply by taking the dot product of both sides with v j. Then, using the properties of the dot product and that { v,..., v k } is orthogonal, for all j, (c v + + c j v j + + c k v k ) v j = x v j (c v ) v j + + (c j v j ) v j + + (c k v k ) v j = x v j c ( v v j ) + + c j ( v j v j ) + + c k ( v k v j ) = x v j c (0) + + c j ( v j ) + + c k (0) = x v j c j ( v j ) = x v j c j = x v j v j. Note that we can divide by v j because v j 0 since v j 0. This is why we assumed the vectors were nonzero in definition of orthogonality. Since we started with an arbitrary solution and found only possibility, this is the only solution if there is one. The term Corollary means an immediate consequence. One consequence of Proposition.8 is that orthogonal sets are lineary independent. Corollary.9. Any orthogonal set of vectors in R n is linearly independent. Proof. Let { v,..., v k } R n be orthogonal. By Proposotion.8, the linear system c v + + c k v k = 0. has at most one solution, naemly the trivial solution. Hence, { v,..., v k } is linearly independent. Example : Solve the following linear systems () + x 3 + x 3 = and + x 3 + x 3 3 3 = () Solutions: Recall that the set,, is orthogonal. By Proposition.8, if () has a solution, 3 it is given by 6 3 6 6 6 3 3 c = =, c 3 = = 3, c 3 3 = = 7 3 Note we do not know that this is a solution yet. This is just the only possible solution if there is one. So, let us check if it is: + 7 = 6. 3 3 3 Yes, it is a solution. Thus, c =, c = 3, c 3 = 7, is the only solution to ().
Notice that, 3, 3 is orthogonal as well. By Proposition.8, if () has a solution, it is given by 3 3 3 c = = = 3, c = = 30 = 6, c 3 = = 0 = 3 3 3 Again, all we know is that this is the only solution if there is one. So, let us check if it is: 3/ 3 + 6 3 + 3 3 = 3/ 3/ 7/ No, it is not. Thus, () has no solution. Exercises: () Determine if the following sets are orthogonal. {[ [ 0 (a), 0] ]} (b) {[ [ 3, 3] ]} (c) 0,, 3 3 3 7 (d),, 8 () Add vectors to the following sets to make orthogonal bases for R n, with the appropriate n, if possible. If it is not possible, why not? {[ ]} (a) (b), 3 3 (c) 3, 0 (d), 3 0
6 (e) 3 (3) Solve the following linear systems using Proposition.8. (a) (b) (c) (d) (e) [ 3 ] [ ] 3 + x = [ ] 7 + x = 3 7 + x = 6 7 3 7 + x + x 3 = 0 + x + x 3 0 0 = 3 Problems:. () Prove the properties of the dot product in Proposition... () Prove the properties of length in Proposition.. { } v 3. () Show that if { v,..., v k } is orthogonal, then v,..., v k v k is orthonormal.. () Show that for any y R n, T : R n R given by T ( x) = x y is a linear map, and that any linear map T : R n R is of this form.. () Suppose that { v,..., v k } and { w,..., w m } satisfy v i w j for all i and j. Show that any linear combination of { v,..., v k } is orthogonal to any linear combination of { w,..., w k }. 6. () Suppose that { u,..., u k } is orthogonal, and u k+ is nonzero so that u k+ u,..., u k+ u k. Show that { u,..., u k, u k+ } is orthogonal. 7. () Show that if { u,..., u n } is an orthogonal basis for R n, then for all x R n, x = ( x u ) u + + ( x u n ) u n 8. () Show that x y = x + y x y. 9. () (Pythagorean Theorem) Show that if x y, then x + y = x + y
7 0. () Show that if { u,..., u k } is orthonormal, then c u + + c n u k = c + + c k.. The goal of this problem is to prove the Cauchy-Schwarz Inequality: For any x, y R n, x y x y. Fix x, y R n, and consider the function f : R R given by f(c) = x c y. (a) (3) Find the minimum value of f and where it occurs. (b) () Deduce that x y x y.. () Suppose L : R n R given by L( x) = x satisfies the properties in Proposition.. L need not coincide with the length function from this section! Then, define x, y = x + y x y. Show that the following hold for all x, y, z R n, (i) x, x = x. (ii) x, y = y, x. (iii) x, y = x, y. (iv) x + y, z = x, z + y, z. (v) n x, y = n x, y for all positive integers n. (vi) r x, y = r x, y for all rational scalars r, where r = a b for some integers a, b. (vii) (Requires a bit of Real Analysis) c x, y = c x, y for all scalars c.