Projection Theorem 1

Transcription

1 Projection Theorem 1

2 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality holds trivially. Therefore, assume y θ. For all scalars, we have [ xy, xy] [ x, x] [ y, x] [ x, y] [ y, y] In particular, for [ xy, ]/[ yy, ], we have [, ] [ xx, ] xy, [ yy, ] or [ xy, ] [ xx, ][ yy, ] x y.

3 Cauchy-Schwarz Inequality Proposition. On a pre-hilbert space X the function x [ x, x] is a norm. Proof. The only requirement for a norm which has not already been established is the triangle inequality. For any x, y X, we have xy [ xy, xy] [ x, x] [ x, y] [ y, x] [ y, y] x [ x, y] y. By the Cauchy-Schwarz inequality, this becomes xy x x y y x y. The square root of the above inequality is the desired result. 3

4 Parallelogram Law Lemma. (The Parallelogram Law) In a pre-hilbert space xy xy x y. Proof. The proof is made by direct expansion of the norms in terms of the inner product. 4

5 Proof. Continuity of the Inner Product Lemma. (Continuity of the Inner Product) Suppose that xn x and yn y in a pre-hilbert space. Then x, y x, y. Since the sequence is convergent, it is bounded; say M. Now x n Applying the Cauchy-Schwarz inequality, we obtain Since is bounded, x n x y x y x y x y x yx y x y y x x y,,,,,,,,. n n n n n n n n n x n x, y x, y x y y x x y. n n n n n x, y x, y M y y x x y. n n n n n n 5

6 Projection Theorem The concept of orthogonality has many of the consequences in pre-hilbert spaces that it has in plane geometry. For example, the Pythagorean theorem is true in pre-hilbert spaces. Lemma. If x y, then xy x y. Proof. xy x+y, x+y x x, y y, x y x y. 6

7 Projection Theorem The optimization problem considered is this: Given a vector x in a pre- Hilbert space X and a subspace M in X, find the vector mm closest to x in the sense that it minimizes x-m. Of course, if x itself lies in M, the solution is trivial. In general, however, three important questions must be answered for a complete solution to the problem. First, is there a vector mm which minimizes x-m, or is there no m that is at least as good as all others? Second, is the solution unique? And third, what is the solution or how is it characterized? We answer these questions now. 7

8 Projection Theorem Theorem 1. Let X be a pre-hilbert space, M a subspace of X, and x an arbitrary vector in X. If there is a vector m M such that x-m x-m for all mm, then m is unique. A necessary and sufficient condition that m M be a unique minimizing vector in M is that the error vector x-m be orthogonal to M. Proof. We show first that if is a minimizing vector, then x-m is m orthogonal to M. Suppose to the contrary that there is an mm which is not orthogonal to x-m. Without loss of generality, we may assume that m 1 and that x m,m. Define the vector m1 in M as m 1 =m δm. 8

9 Then Projection Theorem x-m1 x-m-δm x-m x-m, δm δm, x-m δ. x-m δ x-m Thus, if x m is not orthogonal to M, m is not a minimizing vector. We show now that if x m is orthogonal to M, then m is a unique minimizing vector. For any mm, the Pythagorean theorem gives. x-m x-m m -m x-m m -m Thus, x-m x-m for m m. 9

10 Classical Projection Theorem Theorem. (The Classical Projection Theorem) Let H be a Hilbert space and M a closed subspace of H. Corresponding to any vector xh, there is a unique vector m M such that x m xm for all mm. Furthermore, a necessary and sufficient condition that m M be the unique minimizing vector is that x m be orthogonal to M. Proof. The uniqueness and orthogonality have been established in Theorem 1. It is only required to establish the existence of the minimizing vector. 1

11 Classical Projection Theorem If xm, then m x and everything is settled. Let us assume xm and define δ inf xm. We wish to produce an m M with x m δ. mm For this purpose, let { m } be a sequence of vectors in M such that xm δ. Now, by the parallelogram law, i i ( m x) ( xm ) ( m x) ( xm ) m x xm j i j i j i. Rearranging, we obtain m m m 4 i j j mi mj x x mi x. 11

12 Classical Projection Theorem mi mj For all i, j the vector is in M since is a linear subspace. M mi mj Therefore, by definition of δ, x δ and we obtain j i j i 4 m m m x x m δ. Since m x δ as i, we conclude that i m m as i, j. j i Therefore, { m } is a Cauchy sequence, and since M is a closed subspace i of a complete space, the sequence { } has a limit m in M. By m i continuity of the norm, it follows that x m δ. 1