Projection Theorem 1

Similar documents
CHAPTER II HILBERT SPACES

Another consequence of the Cauchy Schwarz inequality is the continuity of the inner product.

Definitions and Properties of R N

y 2 . = x 1y 1 + x 2 y x + + x n y n 2 7 = 1(2) + 3(7) 5(4) = 3. x x = x x x2 n.

Theorem (4.11). Let M be a closed subspace of a Hilbert space H. For any x, y E, the parallelogram law applied to x/2 and y/2 gives.

Linear Normed Spaces (cont.) Inner Product Spaces

A SYNOPSIS OF HILBERT SPACE THEORY

Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6]

Best approximation in the 2-norm

Inner product spaces. Layers of structure:

Fourier and Wavelet Signal Processing

Linear Algebra Massoud Malek

Some Properties in Generalized n-inner Product Spaces

y 1 y 2 . = x 1y 1 + x 2 y x + + x n y n y n 2 7 = 1(2) + 3(7) 5(4) = 4.

Real Analysis III. (MAT312β) Department of Mathematics University of Ruhuna. A.W.L. Pubudu Thilan

Vector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.

2. Review of Linear Algebra

Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product

Geometric interpretation of signals: background

Orthonormal Bases Fall Consider an inner product space V with inner product f, g and norm

Elementary linear algebra

1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,

Vectors in Function Spaces

The Transpose of a Vector

Inner Product Spaces 6.1 Length and Dot Product in R n

and u and v are orthogonal if and only if u v = 0. u v = x1x2 + y1y2 + z1z2. 1. In R 3 the dot product is defined by

Linear Algebra. Alvin Lin. August December 2017

Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India

Lecture Notes 1: Vector spaces

Lecture 20: 6.1 Inner Products

HILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define

V. SUBSPACES AND ORTHOGONAL PROJECTION

FUNCTIONAL ANALYSIS HAHN-BANACH THEOREM. F (m 2 ) + α m 2 + x 0

Mathematics of Information Spring semester 2018

Section 7.5 Inner Product Spaces

INNER PRODUCT SPACE. Definition 1

Part 1a: Inner product, Orthogonality, Vector/Matrix norm

David Hilbert was old and partly deaf in the nineteen thirties. Yet being a diligent

Functional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...

Homework Assignment #5 Due Wednesday, March 3rd.

Chapter 6 Inner product spaces

Fall TMA4145 Linear Methods. Exercise set 10

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

ABSTRACT CONDITIONAL EXPECTATION IN L 2

Chapter 2. Vectors and Vector Spaces

ON GENERALIZED n-inner PRODUCT SPACES

Math 5520 Homework 2 Solutions

Vector Spaces. Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms.

Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.

This pre-publication material is for review purposes only. Any typographical or technical errors will be corrected prior to publication.

Linear Algebra. Session 12

LINEAR ALGEBRA W W L CHEN

Polar Form of a Complex Number (I)

0, otherwise. Find each of the following limits, or explain that the limit does not exist.

Introduction to Signal Spaces

Inner Product Spaces 5.2 Inner product spaces

MA677 Assignment #3 Morgan Schreffler Due 09/19/12 Exercise 1 Using Hölder s inequality, prove Minkowski s inequality for f, g L p (R d ), p 1:

Lecture 23: 6.1 Inner Products

Applied Linear Algebra in Geoscience Using MATLAB

which has a check digit of 9. This is consistent with the first nine digits of the ISBN, since

6.1. Inner Product, Length and Orthogonality

Cambridge University Press The Mathematics of Signal Processing Steven B. Damelin and Willard Miller Excerpt More information

(, ) : R n R n R. 1. It is bilinear, meaning it s linear in each argument: that is

Notes on Integrable Functions and the Riesz Representation Theorem Math 8445, Winter 06, Professor J. Segert. f(x) = f + (x) + f (x).

Dot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem.

Math 290, Midterm II-key

Math Linear Algebra II. 1. Inner Products and Norms

Exercise Solutions to Functional Analysis

Chapter 8 Gradient Methods

2.4 Hilbert Spaces. Outline

Convex Analysis and Economic Theory Winter 2018

Lecture Note on Linear Algebra 17. Standard Inner Product

Linear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products

Hilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space.

Ordinary Differential Equations II

1.2 LECTURE 2. Scalar Product

Recall that any inner product space V has an associated norm defined by

Applied Linear Algebra in Geoscience Using MATLAB

Inner Product Spaces

5 Compact linear operators

Mathematics Department Stanford University Math 61CM/DM Inner products

Contents. Appendix D (Inner Product Spaces) W-51. Index W-63

Elements of Convex Optimization Theory

Hilbert spaces. 1. Cauchy-Schwarz-Bunyakowsky inequality

Matrix Algebra: Vectors

MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products.

Appendix A Functional Analysis

Functional Analysis MATH and MATH M6202

Further Mathematical Methods (Linear Algebra) 2002

Linear Equations and Vectors

I teach myself... Hilbert spaces

4 Hilbert spaces. The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan

The geometry of least squares

Spectral Theory, with an Introduction to Operator Means. William L. Green

Section 6.1. Inner Product, Length, and Orthogonality

(K + L)(c x) = K(c x) + L(c x) (def of K + L) = K( x) + K( y) + L( x) + L( y) (K, L are linear) = (K L)( x) + (K L)( y).

Normed & Inner Product Vector Spaces

19. Principal Stresses

4.4. Orthogonality. Note. This section is awesome! It is very geometric and shows that much of the geometry of R n holds in Hilbert spaces.

Transcription:

Projection Theorem 1

Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality holds trivially. Therefore, assume y θ. For all scalars, we have [ xy, xy] [ x, x] [ y, x] [ x, y] [ y, y] In particular, for [ xy, ]/[ yy, ], we have [, ] [ xx, ] xy, [ yy, ] or [ xy, ] [ xx, ][ yy, ] x y.

Cauchy-Schwarz Inequality Proposition. On a pre-hilbert space X the function x [ x, x] is a norm. Proof. The only requirement for a norm which has not already been established is the triangle inequality. For any x, y X, we have xy [ xy, xy] [ x, x] [ x, y] [ y, x] [ y, y] x [ x, y] y. By the Cauchy-Schwarz inequality, this becomes xy x x y y x y. The square root of the above inequality is the desired result. 3

Parallelogram Law Lemma. (The Parallelogram Law) In a pre-hilbert space xy xy x y. Proof. The proof is made by direct expansion of the norms in terms of the inner product. 4

Proof. Continuity of the Inner Product Lemma. (Continuity of the Inner Product) Suppose that xn x and yn y in a pre-hilbert space. Then x, y x, y. Since the sequence is convergent, it is bounded; say M. Now x n Applying the Cauchy-Schwarz inequality, we obtain Since is bounded, x n x y x y x y x y x yx y x y y x x y,,,,,,,,. n n n n n n n n n x n x, y x, y x y y x x y. n n n n n x, y x, y M y y x x y. n n n n n n 5

Projection Theorem The concept of orthogonality has many of the consequences in pre-hilbert spaces that it has in plane geometry. For example, the Pythagorean theorem is true in pre-hilbert spaces. Lemma. If x y, then xy x y. Proof. xy x+y, x+y x x, y y, x y x y. 6

Projection Theorem The optimization problem considered is this: Given a vector x in a pre- Hilbert space X and a subspace M in X, find the vector mm closest to x in the sense that it minimizes x-m. Of course, if x itself lies in M, the solution is trivial. In general, however, three important questions must be answered for a complete solution to the problem. First, is there a vector mm which minimizes x-m, or is there no m that is at least as good as all others? Second, is the solution unique? And third, what is the solution or how is it characterized? We answer these questions now. 7

Projection Theorem Theorem 1. Let X be a pre-hilbert space, M a subspace of X, and x an arbitrary vector in X. If there is a vector m M such that x-m x-m for all mm, then m is unique. A necessary and sufficient condition that m M be a unique minimizing vector in M is that the error vector x-m be orthogonal to M. Proof. We show first that if is a minimizing vector, then x-m is m orthogonal to M. Suppose to the contrary that there is an mm which is not orthogonal to x-m. Without loss of generality, we may assume that m 1 and that x m,m. Define the vector m1 in M as m 1 =m δm. 8

Then Projection Theorem x-m1 x-m-δm x-m x-m, δm δm, x-m δ. x-m δ x-m Thus, if x m is not orthogonal to M, m is not a minimizing vector. We show now that if x m is orthogonal to M, then m is a unique minimizing vector. For any mm, the Pythagorean theorem gives. x-m x-m m -m x-m m -m Thus, x-m x-m for m m. 9

Classical Projection Theorem Theorem. (The Classical Projection Theorem) Let H be a Hilbert space and M a closed subspace of H. Corresponding to any vector xh, there is a unique vector m M such that x m xm for all mm. Furthermore, a necessary and sufficient condition that m M be the unique minimizing vector is that x m be orthogonal to M. Proof. The uniqueness and orthogonality have been established in Theorem 1. It is only required to establish the existence of the minimizing vector. 1

Classical Projection Theorem If xm, then m x and everything is settled. Let us assume xm and define δ inf xm. We wish to produce an m M with x m δ. mm For this purpose, let { m } be a sequence of vectors in M such that xm δ. Now, by the parallelogram law, i i ( m x) ( xm ) ( m x) ( xm ) m x xm j i j i j i. Rearranging, we obtain m m m 4 i j j mi mj x x mi x. 11

Classical Projection Theorem mi mj For all i, j the vector is in M since is a linear subspace. M mi mj Therefore, by definition of δ, x δ and we obtain j i j i 4 m m m x x m δ. Since m x δ as i, we conclude that i m m as i, j. j i Therefore, { m } is a Cauchy sequence, and since M is a closed subspace i of a complete space, the sequence { } has a limit m in M. By m i continuity of the norm, it follows that x m δ. 1

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback