17 Inner products Up until now, we have only examined the properties of vectors and matrices in R n. But normally, when we think of R n, we re really thinking of n-dimensional Euclidean space - that is, R n together with the dot product. Once we have the dot product, or more generally an inner product on R n, we can talk about angles, lengths, distances, etc. Definition: An inner product on R n is a function with the following properties: (, ) : R n R n R 1. It is bilinear, meaning it s linear in each argument: that is (c 1 x 1 + c x, y) = c 1 (x 1, y) + c (x, y), x 1, x, y, c 1, c. and (x, c 1 y 1 + c y ) = c 1 (x, y 1 ) + c (x, y ), x, y 1, y, c 1, c.. It is symmetric: (x, y) = (y, x), x, y R n. 3. It is non-degenerate: If (x, y) = 0, y R n, then x = 0. These three properties define a general inner product. Some inner products, like the dot product, have another property: The inner product is said to be positive definite if, in addition to the above,. (x, x) > 0 whenever x 0. Remark: An inner product is also known as a scalar product (because the inner product of two vectors is a scalar). Examples of inner products The dot product in R n given in the standard basis by (x, y) = x y = x 1 y 1 + x y + + x n y n 1
The dot product is positive definite - all four of the properties above hold (exercise). R n with the dot product as an inner product is called n-dimensional Euclidean space, and is denoted E n. In R, with coordinates t, x, y, z, we can define (v 1, v ) = t 1 t x 1 x y 1 y z 1 z. This is an inner product too, since it satisfies (1) - (3) in the definition. But for x = (1, 1, 0, 0) t, we have (x, x) = 0, and for x = (1,, 0, 0), (x, x) = 1 = 3, so it s not positive definite. R with this inner product is called Minkowski space. It is the spacetime of special relativity (invented by Einstein in 1905, and made into a nice geometric space by Minkowski several years later). It is denoted M, and if time permits, we ll look more closely at this space later in the course. Definition: A square matrix G is said to be symmetric if G = G t. Let G be an n n non-singular (det(g) 0) symmetric matrix. Define (x, y) G = x t Gy. It is not difficult to verify that this satisfies the properties in the definition. For example, if (x, y) G = x t Gy = 0 y, then x t G = 0, because if we write x t G as the row vector (a 1, a,..., a n ), then x t Ge 1 = 0 a 1 = 0, x t Ge = 0 a = 0, etc. So all the components of x t G are 0 and hence x t G = 0. Now taking transposes, we find that G t x = Gx = 0. Since G is nonsingular by definition, this means that x = 0, (otherwise the homogeneous system Gx = 0 would have non-trivial solutions and G would be singular) and the inner product is non-degenerate. You should verify that the other two properties hold as well. In fact, any inner product on R n can be written in this form for a suitable matrix G:
x y = x t Gy with G = I. For instance, if 3 x =, and y = 1 1, then x y = x t Iy = x t y = (3,, 1) 1 = 3 + + = 5 The Minkowski inner product has the form x t Gy with G = Diag(1, 1, 1, 1): 1 0 0 0 t t 0 1 0 0 x (t 1, x 1, y 1, z 1 ) x = (t 1, x 1, y 1, z 1 ) = t 1 t x 1 x y 1 y z 1 z. 0 0 1 0 y y 0 0 0 1 z z Exercise: ** Show that under a change of basis given by the matrix E, the matrix G of the inner product becomes G e = E t GE. This is different from the way in which an ordinary matrix (which can be viewed as a linear transformation) behaves. Thus the matrix representing an inner product is a different object from that representing a linear transformation. (Hint: We must have x t Gy = x t eg e y e. Since you know what x e and y e are, plug them in and solve for G e.) For instance, if G = I, so that x y = x t Iy, and E = 1 3, then x y = x t EG E y E, with G E = 10. 3 1 10 Exercise: ** A matrix E is said to preserve the inner product if G e = E t GE = G. This means that the recipe or formula for computing the inner product doesn t change when you pass to the new coordinate system. In E, find the set of all matrices that preserve the dot product. 3
17.1 Euclidean space We now restrict attention to the Euclidean space E n. We ll always be using the dot product, whether we write it as x y or (x, y). Definition: The norm of the vector x is defined by x = x x. In the standard coordinates, this is equal to ( n ) 1/ x = x i. i=1 Example: If x = 1, then x = ( ) + + 1 = 1 Proposition: x > 0 if x 0. cx = c x, c R. Proof: Exercise As you know, x is the distance from the origin 0 to the point x. Or it s the length of the vector x. (Same thing.) The next few properties all follow from the law of cosines: For a triangle with sides a, b, and c, and angles opposite these sides of A, B, and C, c = a + b ab cos(c).
This reduces to Pythagoras theorem if C is a right angle, of course. In the present context, we imagine two vectors x and y with their tails located at 0. The vector going from the tip of y to the tip of x is x y. If θ is the angle between x and y, then the law of cosines reads x y = x + y x y cos θ. (1) On the other hand, from the definition of the norm, we have x y = (x y) (x y) = x x x y y x + y y or x y = x + y x y () Comparing (1) and (), we conclude that x y = cos θ x y, or cos θ = x y x y (3) Since cos θ 1, taking absolute values we get Theorem: x y x y 1, or x y x y () The inequality () is known as the Cauchy-Schwarz inequality. And equation (3) can be taken as the definition of the cosine. Exercises: 1. Find the angle θ between the two vectors v = (1, 0, 1) t and (, 1, 3) t.. When does x y = x y? What is θ when x y = 0? Using the Cauchy-Schwarz inequality, we (i.e., you) can prove the triangle inequality: Theorem: For all x, y, x + y x + y. Proof: Exercise* (Hint: Expand the dot product x + y Cauchy-Schwarz inequality, and take the square root.) = (x + y) (x + y), use the 5
Exercise: The triangle inequality as it s usually encountered in geometry courses states that, in ABC, the distance from A to B is the distance from A to C plus the distance from C to B. Is this the same thing? 6