6 Inner Product and Hilbert Spaces

Transcription

1 6 Inner Product and Hilbert Spaces 6. Motivation Of the different p-norms on R n, p = 2 is special. This is because the 2-norm (λ, λ 2,..., λ n ) 2 = λ 2 + λ λ2 n comes from the an inner product (λ, λ 2,..., λ n ), (µ, µ 2,..., µ n ) = λ µ + λ 2 µ λ n µ n Thus, one can represent x 2 = x, x. It should also be noted that x y x, y = 6.2 Definition () An inner product on a vector space V over R is a mapping, : V V R with the properties: (i) positive definiteness x, x x V x, x x = (ii) symmetry x, y = y, x x, y V (iii) bilinearity αx + βy, z = α x, z + β y, z α, β R, x, y V It should be noted that it can be shown that x, x = x = as follows: x, x = x = by positive-definiteness, = x, = x, = by bilinearity And so x = x, x = Also, one can find a general form of αa + βb, γc + δd as follows: αa + βb, γc + δd = α a, γc + δd + β b, γc + δd by bilinearity = α γc + δd, a + β γc + δd, b by symmetry = α (γ c, a + δ d, a ) + β (γ c, b + δ d, b ) by bilinearity = αγ a, c + αδ a, d + βγ b, c + βδ b, d by symmetry

2 (2) Given an inner product on V, we define a norm x = x, x To prove that this is a norm, we must verify: (i) Positive definiteness This holds by the positive definiteness of the inner product (ii) Linearity with respect to scalar multiplication λx = λx, λx = λ 2 x, x = λ x, x = λ x (iii) The triangle inequality This proof shall be left until we prove the Cauchy-Schwarz-Buniakowsky inequality 6.3 Examples λ R, x V () The vector space R n with inner product (λ,..., λ n ), (µ,..., µ n ) = λ µ + + λ n µ n Proof: (x = (λ, λ 2,..., λ n ), y = (µ, µ 2,..., µ n ), z = (ν, ν 2,..., ν n )) (i) Positive definiteness x, x = (λ, λ 2,..., λ n ), (λ, λ 2,..., λ n ) = λ 2 + λ λ2 n Furthermore, it is clear that x, x = if and only if λ = λ 2 = = λ n = x = (ii) Symmetry (iii) Bilinearity x, y = λ i µ i = i= µ i λ i = y, x x, y R n i= αx + βy, z = (αλ + βµ,..., αλ n + βµ n ), (ν, ν 2,..., ν n ) = (αλ i + βµ i )(ν i ) = (αλ i ν i ) + (βµ i ν i ) = α i= (λ i ν i ) + β i= i= (µ i ν i ) i= i= = α x, z + β y, z α, β R, x, y, z R n (2) The vector space C([, ], R) of continuous functions f : [, ] R with inner product f, g = f(t)g(t)dt Proof: 2

3 (i) Positive definiteness (ii) Symmetry f, g = f, f = (iii) Bilinearity f(t)g(t)dt = αf + βg, h = f(t) 2 dt = = α dt = f C([, ], R) g(t)f(t)dt = g, f f, g C([, ], R) (αf + βg)(t)h(t)dt (αf(t)h(t) + βg(t)h(t))dt f(t)h(t)dt + β = α f, h + β g, h g(t)h(t)dt It is also interesting to note that the norm defined by this inner product is not the sup-norm, f = sup f(t) t [,] but the 2-norm, f 2 = ( f, f = ) f(t) 2 2 dt (3) The vector space R 3 with inner product, A defined by x, y A = xay T x, y R where A = Proof: (i) Positive definiteness We note that for a Hermitian matrix (symmetric in the real case), the condition xax T > x R 3 is satisfied if and only if A is positive definite (by definition of positive definite matrices). Recalling that an n n-matrix is positive definite if and only if its n eigenvalues are positive, it is simple to confirm that χ A (x) = x 3 9x 2 + 2x 3, and a quick check reveals that χ A (λ) = for three values of λ >. As a result, we can confirm that, A is positivedefinite. 3

4 5 x y 5 Figure : A plot of χ A (x) (ii) Symmetry We begin by noting that (xa) T = A T x T (iii) Bilinearity x, y A = xay T = x ( ya T ) T = x (ya) T A = A T = (ya) x T = yax T = y, x A αx + βy, z A = (αx + βy) Az T = αxaz T + βyaz T = α x, z A + β y, z A Noting that this proof is actually quite general, we can draw the conclusion that, A defines an inner product on R n if and only if A is an n n symmetric matrix with n positive eigenvalues. We know that for the standard dot product on R n, it is the case that x, y = x y cos θ, where θ is the angle between the vectors x and y. Thus, x, y = x y cos θ x y. One can prove that this is true of all inner product spaces. 4

5 6.4 The Cauchy-Schwarz-Buniakowsky Inequality Let, be an inner product on a vector space V. Then we have x, y x y x, y V and equality holds if and only if x and y are co-linear (that is y = or x = αy for some α R). Proof If x =, then x, y = y, y = y, y =, and so the proof is complete (as x = x = ). Similarly, if y =, then x, y = x, x = x, x =. Thus we can assume that x, y. Consider x ty V for t R. x ty, x ty = x, x t x, y t y, x +t 2 y, y = x 2 2t x, y +t 2 y 2 Since t was arbitrary, we can choose what we want it to be. So let t = ± x y, which is always defined and non-zero, since x, y. Substituting this into the above, we have ±2 x x, y 2 x 2 y ±2 x, y 2 x y x, y x y For equality to hold, we neet x ty = by positive definiteness. x = ty, and so x and y are co-linear. Thus 6.5 Corollary We can now prove that the norm defined by the inner product is in fact a norm, by proving the triangle inequality. So let, be an inner product on a vector space V. Then x = x, x is a norm on V. Proof All that remains is to verify the triangle inequality. x + y 2 = x + y, x + y = x, x + x, y + y, x + y, y = x, x + y, y + 2 x, y x 2 + y x y by (6.4) = ( x + y ) 2 x, y V but x, and f(x) = x 2 is monotonically increasing for x, so x + y x + y x, y V This completes the proof that =, is a norm. 5

6 6.6 Corollary Let V be an inner product space with an induced norm. Then, : V V R is continuous, that is if lim n x n = x and lim n y n = y, then lim n x n, y n = x, y. Proof x n, y n x, y = x n, y n x, y n + x, y n x, y x n, y n x, y n + x, y n x, y = x n x, y n + x, y n y x n x y n + x y n y ( { }) But lim y ε n = y, so ε > N N so that n N, y n y < min n 2 x, ε (because x is constant, and thus so is 2 x ). Since y n y <, we know that y n < y + n N. Likewise, by a similar calculation as above, we know that ε >, n M, x n x < ε 2( y +), since y is again constant. Thus, we can say that n max {N, M} { } ε ε x n, y n x, y < ( y + ) + x min 2( y + ) 2 x, < ε 2 + ε 2 = ε Thus, ε > ( N N such that x n, y n x, y < ε). proof that, is continuous. This completes the 6.7 Results () Since f, g = f(t)g(t)dt is an inner product on (C([, ], R), (6.4) gives us that ( ) ( f(t)g(t)dt f(t) 2 2 ) dt g(t) 2 2 dt f, g (C([, ], R) ( (2) (6.6) implies that f 2 = f(t)2 dt defines a norm on (C([, ], R). This is not the sup-norm, and is not equivalent to the sup-norm. We do have that f 2 2 = f(t) 2 dt sup t [,] ) 2 f(t) 2 dt = sup f(t) 2 = f 2 t [,] However, the other direction ( f c f 2 ) is not true. Consider f n (t) = t n, f n (C([, ], R) n N. We note that f n () =, f n (t) = n N. Thus f n = n N f n 2 = t 2n dt = 2n + t2n+ t= t= = 2n + 6

7 So assume c such that f c f 2 n N. Then we merely need to choose n > c 2, and c f n 2 = c 2n + < c 2n < c 2c < = f n Thus 2 is weaker than, that is a sequence which converges with respect to the norm will always converge with respect to the norm 2, but the converse is not necessarily true. (3) This last point brings up the question of completeness with respect to the 2-norm. Consider the sequence of functions {f n } n N, where t 2 n f n (t) = nt n n < t < 2 2 t If this function is to converge, it must converge to the function t.6.8 Figure 2: A plot of f 2 (t), f 5 (t), and f 5 (t) f(t) = { t < 2 t 2 But this function is clearly not continuous, and so is not an element of C([, ], R). Assume n m without loss of generality. (i) We first verify that this example does not cause a problem for the sup-norm. f n f m = sup f n (t) f m (t) t [,] = sup t [,] = n m t 2 n nt n n < t 2 m (n m)t n m 2 2 m < t < 2 2 t 7

8 So, given ε = 4, and any N N, one can choose n = N, and m = 2N, giving f n f m = 2. So this is not a Cauchy sequence with respect to the sup-norm, and will thus not converge. (ii) We now show that this sequence does converge (and in fact does converge to f) with respect to the 2-norm. 6.8 Definition f n f 2 2 = f n (t) f(t) 2 dt = 2 2 n = 3n = 3n ( nt n 2 + ) 2 dt [ ( nt n 2 + ) 3 ] t= 2 t= 2 n So given ε >, we can take N > 3ε 2, and find that for every n N, f n f 2 < ε Thus, proving that the sequence {f n } n N converges to f, and that C([, ], R) is not complete under the 2-norm. An inner product space that is complete with respect to the induced norm is called a Hilbert Space. 8