Projection Theorem 1

Size: px
Start display at page:

Download "Projection Theorem 1"

Transcription

1 Projection Theorem 1

2 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality holds trivially. Therefore, assume y θ. For all scalars, we have [ xy, xy] [ x, x] [ y, x] [ x, y] [ y, y] In particular, for [ xy, ]/[ yy, ], we have [, ] [ xx, ] xy, [ yy, ] or [ xy, ] [ xx, ][ yy, ] x y.

3 Cauchy-Schwarz Inequality Proposition. On a pre-hilbert space X the function x [ x, x] is a norm. Proof. The only requirement for a norm which has not already been established is the triangle inequality. For any x, y X, we have xy [ xy, xy] [ x, x] [ x, y] [ y, x] [ y, y] x [ x, y] y. By the Cauchy-Schwarz inequality, this becomes xy x x y y x y. The square root of the above inequality is the desired result. 3

4 Parallelogram Law Lemma. (The Parallelogram Law) In a pre-hilbert space xy xy x y. Proof. The proof is made by direct expansion of the norms in terms of the inner product. 4

5 Proof. Continuity of the Inner Product Lemma. (Continuity of the Inner Product) Suppose that xn x and yn y in a pre-hilbert space. Then x, y x, y. Since the sequence is convergent, it is bounded; say M. Now x n Applying the Cauchy-Schwarz inequality, we obtain Since is bounded, x n x y x y x y x y x yx y x y y x x y,,,,,,,,. n n n n n n n n n x n x, y x, y x y y x x y. n n n n n x, y x, y M y y x x y. n n n n n n 5

6 Projection Theorem The concept of orthogonality has many of the consequences in pre-hilbert spaces that it has in plane geometry. For example, the Pythagorean theorem is true in pre-hilbert spaces. Lemma. If x y, then xy x y. Proof. xy x+y, x+y x x, y y, x y x y. 6

7 Projection Theorem The optimization problem considered is this: Given a vector x in a pre- Hilbert space X and a subspace M in X, find the vector mm closest to x in the sense that it minimizes x-m. Of course, if x itself lies in M, the solution is trivial. In general, however, three important questions must be answered for a complete solution to the problem. First, is there a vector mm which minimizes x-m, or is there no m that is at least as good as all others? Second, is the solution unique? And third, what is the solution or how is it characterized? We answer these questions now. 7

8 Projection Theorem Theorem 1. Let X be a pre-hilbert space, M a subspace of X, and x an arbitrary vector in X. If there is a vector m M such that x-m x-m for all mm, then m is unique. A necessary and sufficient condition that m M be a unique minimizing vector in M is that the error vector x-m be orthogonal to M. Proof. We show first that if is a minimizing vector, then x-m is m orthogonal to M. Suppose to the contrary that there is an mm which is not orthogonal to x-m. Without loss of generality, we may assume that m 1 and that x m,m. Define the vector m1 in M as m 1 =m δm. 8

9 Then Projection Theorem x-m1 x-m-δm x-m x-m, δm δm, x-m δ. x-m δ x-m Thus, if x m is not orthogonal to M, m is not a minimizing vector. We show now that if x m is orthogonal to M, then m is a unique minimizing vector. For any mm, the Pythagorean theorem gives. x-m x-m m -m x-m m -m Thus, x-m x-m for m m. 9

10 Classical Projection Theorem Theorem. (The Classical Projection Theorem) Let H be a Hilbert space and M a closed subspace of H. Corresponding to any vector xh, there is a unique vector m M such that x m xm for all mm. Furthermore, a necessary and sufficient condition that m M be the unique minimizing vector is that x m be orthogonal to M. Proof. The uniqueness and orthogonality have been established in Theorem 1. It is only required to establish the existence of the minimizing vector. 1

11 Classical Projection Theorem If xm, then m x and everything is settled. Let us assume xm and define δ inf xm. We wish to produce an m M with x m δ. mm For this purpose, let { m } be a sequence of vectors in M such that xm δ. Now, by the parallelogram law, i i ( m x) ( xm ) ( m x) ( xm ) m x xm j i j i j i. Rearranging, we obtain m m m 4 i j j mi mj x x mi x. 11

12 Classical Projection Theorem mi mj For all i, j the vector is in M since is a linear subspace. M mi mj Therefore, by definition of δ, x δ and we obtain j i j i 4 m m m x x m δ. Since m x δ as i, we conclude that i m m as i, j. j i Therefore, { m } is a Cauchy sequence, and since M is a closed subspace i of a complete space, the sequence { } has a limit m in M. By m i continuity of the norm, it follows that x m δ. 1

CHAPTER II HILBERT SPACES

CHAPTER II HILBERT SPACES CHAPTER II HILBERT SPACES 2.1 Geometry of Hilbert Spaces Definition 2.1.1. Let X be a complex linear space. An inner product on X is a function, : X X C which satisfies the following axioms : 1. y, x =

More information

Another consequence of the Cauchy Schwarz inequality is the continuity of the inner product.

Another consequence of the Cauchy Schwarz inequality is the continuity of the inner product. . Inner product spaces 1 Theorem.1 (Cauchy Schwarz inequality). If X is an inner product space then x,y x y. (.) Proof. First note that 0 u v v u = u v u v Re u,v. (.3) Therefore, Re u,v u v (.) for all

More information

Definitions and Properties of R N

Definitions and Properties of R N Definitions and Properties of R N R N as a set As a set R n is simply the set of all ordered n-tuples (x 1,, x N ), called vectors. We usually denote the vector (x 1,, x N ), (y 1,, y N ), by x, y, or

More information

y 2 . = x 1y 1 + x 2 y x + + x n y n 2 7 = 1(2) + 3(7) 5(4) = 3. x x = x x x2 n.

y 2 . = x 1y 1 + x 2 y x + + x n y n 2 7 = 1(2) + 3(7) 5(4) = 3. x x = x x x2 n. 6.. Length, Angle, and Orthogonality In this section, we discuss the defintion of length and angle for vectors and define what it means for two vectors to be orthogonal. Then, we see that linear systems

More information

Theorem (4.11). Let M be a closed subspace of a Hilbert space H. For any x, y E, the parallelogram law applied to x/2 and y/2 gives.

Theorem (4.11). Let M be a closed subspace of a Hilbert space H. For any x, y E, the parallelogram law applied to x/2 and y/2 gives. Math 641 Lecture #19 4.11, 4.12, 4.13 Inner Products and Linear Functionals (Continued) Theorem (4.10). Every closed, convex set E in a Hilbert space H contains a unique element of smallest norm, i.e.,

More information

Linear Normed Spaces (cont.) Inner Product Spaces

Linear Normed Spaces (cont.) Inner Product Spaces Linear Normed Spaces (cont.) Inner Product Spaces October 6, 017 Linear Normed Spaces (cont.) Theorem A normed space is a metric space with metric ρ(x,y) = x y Note: if x n x then x n x, and if {x n} is

More information

A SYNOPSIS OF HILBERT SPACE THEORY

A SYNOPSIS OF HILBERT SPACE THEORY A SYNOPSIS OF HILBERT SPACE THEORY Below is a summary of Hilbert space theory that you find in more detail in any book on functional analysis, like the one Akhiezer and Glazman, the one by Kreiszig or

More information

Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold: Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v

More information

ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6]

ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] Inner products and Norms Inner product or dot product of 2 vectors u and v in R n : u.v = u 1 v 1 + u 2 v 2 + + u n v n Calculate u.v when u = 1 2 2 0 v = 1 0

More information

Inner product spaces. Layers of structure:

Inner product spaces. Layers of structure: Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty

More information

Best approximation in the 2-norm

Best approximation in the 2-norm Best approximation in the 2-norm Department of Mathematical Sciences, NTNU september 26th 2012 Vector space A real vector space V is a set with a 0 element and three operations: Addition: x, y V then x

More information

Fourier and Wavelet Signal Processing

Fourier and Wavelet Signal Processing Ecole Polytechnique Federale de Lausanne (EPFL) Audio-Visual Communications Laboratory (LCAV) Fourier and Wavelet Signal Processing Martin Vetterli Amina Chebira, Ali Hormati Spring 2011 2/25/2011 1 Outline

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Some Properties in Generalized n-inner Product Spaces

Some Properties in Generalized n-inner Product Spaces Int. Journal of Math. Analysis, Vol. 4, 2010, no. 45, 2229-2234 Some Properties in Generalized n-inner Product Spaces B. Surender Reddy Department of Mathematics, PGCS, Saifabad Osmania University, Hyderabad-500004,

More information

y 1 y 2 . = x 1y 1 + x 2 y x + + x n y n y n 2 7 = 1(2) + 3(7) 5(4) = 4.

y 1 y 2 . = x 1y 1 + x 2 y x + + x n y n y n 2 7 = 1(2) + 3(7) 5(4) = 4. . Length, Angle, and Orthogonality In this section, we discuss the defintion of length and angle for vectors. We also define what it means for two vectors to be orthogonal. Then we see that linear systems

More information

Real Analysis III. (MAT312β) Department of Mathematics University of Ruhuna. A.W.L. Pubudu Thilan

Real Analysis III. (MAT312β) Department of Mathematics University of Ruhuna. A.W.L. Pubudu Thilan Real Analysis III (MAT312β) Department of Mathematics University of Ruhuna A.W.L. Pubudu Thilan Department of Mathematics University of Ruhuna Real Analysis III(MAT312β) 1/87 About course unit Course unit:

More information

Vector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.

Vector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis. Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar

More information

2. Review of Linear Algebra

2. Review of Linear Algebra 2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

More information

Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product

Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )

More information

Geometric interpretation of signals: background

Geometric interpretation of signals: background Geometric interpretation of signals: background David G. Messerschmitt Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-006-9 http://www.eecs.berkeley.edu/pubs/techrpts/006/eecs-006-9.html

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

Orthonormal Bases Fall Consider an inner product space V with inner product f, g and norm

Orthonormal Bases Fall Consider an inner product space V with inner product f, g and norm 8.03 Fall 203 Orthonormal Bases Consider an inner product space V with inner product f, g and norm f 2 = f, f Proposition (Continuity) If u n u 0 and v n v 0 as n, then u n u ; u n, v n u, v. Proof. Note

More information

1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,

1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e., Abstract Hilbert Space Results We have learned a little about the Hilbert spaces L U and and we have at least defined H 1 U and the scale of Hilbert spaces H p U. Now we are going to develop additional

More information

Vectors in Function Spaces

Vectors in Function Spaces Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also

More information

The Transpose of a Vector

The Transpose of a Vector 8 CHAPTER Vectors The Transpose of a Vector We now consider the transpose of a vector in R n, which is a row vector. For a vector u 1 u. u n the transpose is denoted by u T = [ u 1 u u n ] EXAMPLE -5 Find

More information

Inner Product Spaces 6.1 Length and Dot Product in R n

Inner Product Spaces 6.1 Length and Dot Product in R n Inner Product Spaces 6.1 Length and Dot Product in R n Summer 2017 Goals We imitate the concept of length and angle between two vectors in R 2, R 3 to define the same in the n space R n. Main topics are:

More information

and u and v are orthogonal if and only if u v = 0. u v = x1x2 + y1y2 + z1z2. 1. In R 3 the dot product is defined by

and u and v are orthogonal if and only if u v = 0. u v = x1x2 + y1y2 + z1z2. 1. In R 3 the dot product is defined by Linear Algebra [] 4.2 The Dot Product and Projections. In R 3 the dot product is defined by u v = u v cos θ. 2. For u = (x, y, z) and v = (x2, y2, z2), we have u v = xx2 + yy2 + zz2. 3. cos θ = u v u v,

More information

Linear Algebra. Alvin Lin. August December 2017

Linear Algebra. Alvin Lin. August December 2017 Linear Algebra Alvin Lin August 207 - December 207 Linear Algebra The study of linear algebra is about two basic things. We study vector spaces and structure preserving maps between vector spaces. A vector

More information

Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India

Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India CHAPTER 9 BY Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India E-mail : mantusaha.bu@gmail.com Introduction and Objectives In the preceding chapters, we discussed normed

More information

Lecture Notes 1: Vector spaces

Lecture Notes 1: Vector spaces Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector

More information

Lecture 20: 6.1 Inner Products

Lecture 20: 6.1 Inner Products Lecture 0: 6.1 Inner Products Wei-Ta Chu 011/1/5 Definition An inner product on a real vector space V is a function that associates a real number u, v with each pair of vectors u and v in V in such a way

More information

HILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define

HILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define HILBERT SPACES AND THE RADON-NIKODYM THEOREM STEVEN P. LALLEY 1. DEFINITIONS Definition 1. A real inner product space is a real vector space V together with a symmetric, bilinear, positive-definite mapping,

More information

V. SUBSPACES AND ORTHOGONAL PROJECTION

V. SUBSPACES AND ORTHOGONAL PROJECTION V. SUBSPACES AND ORTHOGONAL PROJECTION In this chapter we will discuss the concept of subspace of Hilbert space, introduce a series of subspaces related to Haar wavelet, explore the orthogonal projection

More information

FUNCTIONAL ANALYSIS HAHN-BANACH THEOREM. F (m 2 ) + α m 2 + x 0

FUNCTIONAL ANALYSIS HAHN-BANACH THEOREM. F (m 2 ) + α m 2 + x 0 FUNCTIONAL ANALYSIS HAHN-BANACH THEOREM If M is a linear subspace of a normal linear space X and if F is a bounded linear functional on M then F can be extended to M + [x 0 ] without changing its norm.

More information

Mathematics of Information Spring semester 2018

Mathematics of Information Spring semester 2018 Communication Technology Laboratory Prof. Dr. H. Bölcskei Sternwartstrasse 7 CH-809 Zürich Mathematics of Information Spring semester 08 Solution to Homework Problem Overcomplete expansion in R a) Consider

More information

Section 7.5 Inner Product Spaces

Section 7.5 Inner Product Spaces Section 7.5 Inner Product Spaces With the dot product defined in Chapter 6, we were able to study the following properties of vectors in R n. ) Length or norm of a vector u. ( u = p u u ) 2) Distance of

More information

INNER PRODUCT SPACE. Definition 1

INNER PRODUCT SPACE. Definition 1 INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of

More information

Part 1a: Inner product, Orthogonality, Vector/Matrix norm

Part 1a: Inner product, Orthogonality, Vector/Matrix norm Part 1a: Inner product, Orthogonality, Vector/Matrix norm September 19, 2018 Numerical Linear Algebra Part 1a September 19, 2018 1 / 16 1. Inner product on a linear space V over the number field F A map,

More information

David Hilbert was old and partly deaf in the nineteen thirties. Yet being a diligent

David Hilbert was old and partly deaf in the nineteen thirties. Yet being a diligent Chapter 5 ddddd dddddd dddddddd ddddddd dddddddd ddddddd Hilbert Space The Euclidean norm is special among all norms defined in R n for being induced by the Euclidean inner product (the dot product). A

More information

Functional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...

Functional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability... Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................

More information

Homework Assignment #5 Due Wednesday, March 3rd.

Homework Assignment #5 Due Wednesday, March 3rd. Homework Assignment #5 Due Wednesday, March 3rd. 1. In this problem, X will be a separable Banach space. Let B be the closed unit ball in X. We want to work out a solution to E 2.5.3 in the text. Work

More information

Chapter 6 Inner product spaces

Chapter 6 Inner product spaces Chapter 6 Inner product spaces 6.1 Inner products and norms Definition 1 Let V be a vector space over F. An inner product on V is a function, : V V F such that the following conditions hold. x+z,y = x,y

More information

Fall TMA4145 Linear Methods. Exercise set 10

Fall TMA4145 Linear Methods. Exercise set 10 Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 207 Exercise set 0 Please justify your answers! The most important part is how you arrive at

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

ABSTRACT CONDITIONAL EXPECTATION IN L 2

ABSTRACT CONDITIONAL EXPECTATION IN L 2 ABSTRACT CONDITIONAL EXPECTATION IN L 2 Abstract. We prove that conditional expecations exist in the L 2 case. The L 2 treatment also gives us a geometric interpretation for conditional expectation. 1.

More information

Chapter 2. Vectors and Vector Spaces

Chapter 2. Vectors and Vector Spaces 2.1. Operations on Vectors 1 Chapter 2. Vectors and Vector Spaces Section 2.1. Operations on Vectors Note. In this section, we define several arithmetic operations on vectors (especially, vector addition

More information

ON GENERALIZED n-inner PRODUCT SPACES

ON GENERALIZED n-inner PRODUCT SPACES Novi Sad J Math Vol 41, No 2, 2011, 73-80 ON GENERALIZED n-inner PRODUCT SPACES Renu Chugh 1, Sushma Lather 2 Abstract The primary purpose of this paper is to derive a generalized (n k) inner product with

More information

Math 5520 Homework 2 Solutions

Math 5520 Homework 2 Solutions Math 552 Homework 2 Solutions March, 26. Consider the function fx) = 2x ) 3 if x, 3x ) 2 if < x 2. Determine for which k there holds f H k, 2). Find D α f for α k. Solution. We show that k = 2. The formulas

More information

Vector Spaces. Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms.

Vector Spaces. Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms. Vector Spaces Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms. For each two vectors a, b ν there exists a summation procedure: a +

More information

Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.

Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set. Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an

More information

This pre-publication material is for review purposes only. Any typographical or technical errors will be corrected prior to publication.

This pre-publication material is for review purposes only. Any typographical or technical errors will be corrected prior to publication. This pre-publication material is for review purposes only. Any typographical or technical errors will be corrected prior to publication. Copyright Pearson Canada Inc. All rights reserved. Copyright Pearson

More information

Linear Algebra. Session 12

Linear Algebra. Session 12 Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

More information

LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

More information

Polar Form of a Complex Number (I)

Polar Form of a Complex Number (I) Polar Form of a Complex Number (I) The polar form of a complex number z = x + iy is given by z = r (cos θ + i sin θ) where θ is the angle from the real axes to the vector z and r is the modulus of z, i.e.

More information

0, otherwise. Find each of the following limits, or explain that the limit does not exist.

0, otherwise. Find each of the following limits, or explain that the limit does not exist. Midterm Solutions 1, y x 4 1. Let f(x, y) = 1, y 0 0, otherwise. Find each of the following limits, or explain that the limit does not exist. (a) (b) (c) lim f(x, y) (x,y) (0,1) lim f(x, y) (x,y) (2,3)

More information

Introduction to Signal Spaces

Introduction to Signal Spaces Introduction to Signal Spaces Selin Aviyente Department of Electrical and Computer Engineering Michigan State University January 12, 2010 Motivation Outline 1 Motivation 2 Vector Space 3 Inner Product

More information

Inner Product Spaces 5.2 Inner product spaces

Inner Product Spaces 5.2 Inner product spaces Inner Product Spaces 5.2 Inner product spaces November 15 Goals Concept of length, distance, and angle in R 2 or R n is extended to abstract vector spaces V. Sucn a vector space will be called an Inner

More information

MA677 Assignment #3 Morgan Schreffler Due 09/19/12 Exercise 1 Using Hölder s inequality, prove Minkowski s inequality for f, g L p (R d ), p 1:

MA677 Assignment #3 Morgan Schreffler Due 09/19/12 Exercise 1 Using Hölder s inequality, prove Minkowski s inequality for f, g L p (R d ), p 1: Exercise 1 Using Hölder s inequality, prove Minkowski s inequality for f, g L p (R d ), p 1: f + g p f p + g p. Proof. If f, g L p (R d ), then since f(x) + g(x) max {f(x), g(x)}, we have f(x) + g(x) p

More information

Lecture 23: 6.1 Inner Products

Lecture 23: 6.1 Inner Products Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such

More information

Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

which has a check digit of 9. This is consistent with the first nine digits of the ISBN, since

which has a check digit of 9. This is consistent with the first nine digits of the ISBN, since vector Then the check digit c is computed using the following procedure: 1. Form the dot product. 2. Divide by 11, thereby producing a remainder c that is an integer between 0 and 10, inclusive. The check

More information

6.1. Inner Product, Length and Orthogonality

6.1. Inner Product, Length and Orthogonality These are brief notes for the lecture on Friday November 13, and Monday November 1, 2009: they are not complete, but they are a guide to what I want to say on those days. They are guaranteed to be incorrect..1.

More information

Cambridge University Press The Mathematics of Signal Processing Steven B. Damelin and Willard Miller Excerpt More information

Cambridge University Press The Mathematics of Signal Processing Steven B. Damelin and Willard Miller Excerpt More information Introduction Consider a linear system y = Φx where Φ can be taken as an m n matrix acting on Euclidean space or more generally, a linear operator on a Hilbert space. We call the vector x a signal or input,

More information

(, ) : R n R n R. 1. It is bilinear, meaning it s linear in each argument: that is

(, ) : R n R n R. 1. It is bilinear, meaning it s linear in each argument: that is 17 Inner products Up until now, we have only examined the properties of vectors and matrices in R n. But normally, when we think of R n, we re really thinking of n-dimensional Euclidean space - that is,

More information

Notes on Integrable Functions and the Riesz Representation Theorem Math 8445, Winter 06, Professor J. Segert. f(x) = f + (x) + f (x).

Notes on Integrable Functions and the Riesz Representation Theorem Math 8445, Winter 06, Professor J. Segert. f(x) = f + (x) + f (x). References: Notes on Integrable Functions and the Riesz Representation Theorem Math 8445, Winter 06, Professor J. Segert Evans, Partial Differential Equations, Appendix 3 Reed and Simon, Functional Analysis,

More information

Dot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem.

Dot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem. Dot Products K. Behrend April 3, 008 Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem. Contents The dot product 3. Length of a vector........................

More information

Math 290, Midterm II-key

Math 290, Midterm II-key Math 290, Midterm II-key Name (Print): (first) Signature: (last) The following rules apply: There are a total of 20 points on this 50 minutes exam. This contains 7 pages (including this cover page) and

More information

Math Linear Algebra II. 1. Inner Products and Norms

Math Linear Algebra II. 1. Inner Products and Norms Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,

More information

Exercise Solutions to Functional Analysis

Exercise Solutions to Functional Analysis Exercise Solutions to Functional Analysis Note: References refer to M. Schechter, Principles of Functional Analysis Exersize that. Let φ,..., φ n be an orthonormal set in a Hilbert space H. Show n f n

More information

Chapter 8 Gradient Methods

Chapter 8 Gradient Methods Chapter 8 Gradient Methods An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Introduction Recall that a level set of a function is the set of points satisfying for some constant. Thus, a point

More information

2.4 Hilbert Spaces. Outline

2.4 Hilbert Spaces. Outline 2.4 Hilbert Spaces Tom Lewis Spring Semester 2017 Outline Hilbert spaces L 2 ([a, b]) Orthogonality Approximations Definition A Hilbert space is an inner product space which is complete in the norm defined

More information

Convex Analysis and Economic Theory Winter 2018

Convex Analysis and Economic Theory Winter 2018 Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Supplement A: Mathematical background A.1 Extended real numbers The extended real number

More information

Lecture Note on Linear Algebra 17. Standard Inner Product

Lecture Note on Linear Algebra 17. Standard Inner Product Lecture Note on Linear Algebra 17. Standard Inner Product Wei-Shi Zheng, wszheng@ieee.org, 2011 November 21, 2011 1 What Do You Learn from This Note Up to this point, the theory we have established can

More information

Linear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products

Linear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 6A: Inner products In this chapter, the field F = R or C. We regard F equipped with a conjugation χ : F F. If F =

More information

Hilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space.

Hilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space. Hilbert Spaces Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space. Vector Space. Vector space, ν, over the field of complex numbers,

More information

Ordinary Differential Equations II

Ordinary Differential Equations II Ordinary Differential Equations II February 23 2017 Separation of variables Wave eq. (PDE) 2 u t (t, x) = 2 u 2 c2 (t, x), x2 c > 0 constant. Describes small vibrations in a homogeneous string. u(t, x)

More information

1.2 LECTURE 2. Scalar Product

1.2 LECTURE 2. Scalar Product 6 CHAPTER 1. VECTOR ALGEBRA Pythagean theem. cos 2 α 1 + cos 2 α 2 + cos 2 α 3 = 1 There is a one-to-one crespondence between the components of the vect on the one side and its magnitude and the direction

More information

Recall that any inner product space V has an associated norm defined by

Recall that any inner product space V has an associated norm defined by Hilbert Spaces Recall that any inner product space V has an associated norm defined by v = v v. Thus an inner product space can be viewed as a special kind of normed vector space. In particular every inner

More information

Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

Inner Product Spaces

Inner Product Spaces Inner Product Spaces Introduction Recall in the lecture on vector spaces that geometric vectors (i.e. vectors in two and three-dimensional Cartesian space have the properties of addition, subtraction,

More information

5 Compact linear operators

5 Compact linear operators 5 Compact linear operators One of the most important results of Linear Algebra is that for every selfadjoint linear map A on a finite-dimensional space, there exists a basis consisting of eigenvectors.

More information

Mathematics Department Stanford University Math 61CM/DM Inner products

Mathematics Department Stanford University Math 61CM/DM Inner products Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector

More information

Contents. Appendix D (Inner Product Spaces) W-51. Index W-63

Contents. Appendix D (Inner Product Spaces) W-51. Index W-63 Contents Appendix D (Inner Product Spaces W-5 Index W-63 Inner city space W-49 W-5 Chapter : Appendix D Inner Product Spaces The inner product, taken of any two vectors in an arbitrary vector space, generalizes

More information

Elements of Convex Optimization Theory

Elements of Convex Optimization Theory Elements of Convex Optimization Theory Costis Skiadas August 2015 This is a revised and extended version of Appendix A of Skiadas (2009), providing a self-contained overview of elements of convex optimization

More information

Hilbert spaces. 1. Cauchy-Schwarz-Bunyakowsky inequality

Hilbert spaces. 1. Cauchy-Schwarz-Bunyakowsky inequality (October 29, 2016) Hilbert spaces Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ [This document is http://www.math.umn.edu/ garrett/m/fun/notes 2016-17/03 hsp.pdf] Hilbert spaces are

More information

Matrix Algebra: Vectors

Matrix Algebra: Vectors A Matrix Algebra: Vectors A Appendix A: MATRIX ALGEBRA: VECTORS A 2 A MOTIVATION Matrix notation was invented primarily to express linear algebra relations in compact form Compactness enhances visualization

More information

MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products.

MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products. MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products. Orthogonal projection Theorem 1 Let V be a subspace of R n. Then any vector x R n is uniquely represented

More information

Appendix A Functional Analysis

Appendix A Functional Analysis Appendix A Functional Analysis A.1 Metric Spaces, Banach Spaces, and Hilbert Spaces Definition A.1. Metric space. Let X be a set. A map d : X X R is called metric on X if for all x,y,z X it is i) d(x,y)

More information

Functional Analysis MATH and MATH M6202

Functional Analysis MATH and MATH M6202 Functional Analysis MATH 36202 and MATH M6202 1 Inner Product Spaces and Normed Spaces Inner Product Spaces Functional analysis involves studying vector spaces where we additionally have the notion of

More information

Further Mathematical Methods (Linear Algebra) 2002

Further Mathematical Methods (Linear Algebra) 2002 Further Mathematical Methods (Linear Algebra) 22 Solutions For Problem Sheet 3 In this Problem Sheet, we looked at some problems on real inner product spaces. In particular, we saw that many different

More information

Linear Equations and Vectors

Linear Equations and Vectors Chapter Linear Equations and Vectors Linear Algebra, Fall 6 Matrices and Systems of Linear Equations Figure. Linear Algebra, Fall 6 Figure. Linear Algebra, Fall 6 Figure. Linear Algebra, Fall 6 Unique

More information

I teach myself... Hilbert spaces

I teach myself... Hilbert spaces I teach myself... Hilbert spaces by F.J.Sayas, for MATH 806 November 4, 2015 This document will be growing with the semester. Every in red is for you to justify. Even if we start with the basic definition

More information

4 Hilbert spaces. The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan

4 Hilbert spaces. The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan Wir müssen wissen, wir werden wissen. David Hilbert We now continue to study a special class of Banach spaces,

More information

The geometry of least squares

The geometry of least squares The geometry of least squares We can think of a vector as a point in space, where the elements of the vector are the coordinates of the point. Consider for example, the following vector s: t = ( 4, 0),

More information

Spectral Theory, with an Introduction to Operator Means. William L. Green

Spectral Theory, with an Introduction to Operator Means. William L. Green Spectral Theory, with an Introduction to Operator Means William L. Green January 30, 2008 Contents Introduction............................... 1 Hilbert Space.............................. 4 Linear Maps

More information

Section 6.1. Inner Product, Length, and Orthogonality

Section 6.1. Inner Product, Length, and Orthogonality Section 6. Inner Product, Length, and Orthogonality Orientation Almost solve the equation Ax = b Problem: In the real world, data is imperfect. x v u But due to measurement error, the measured x is not

More information

(K + L)(c x) = K(c x) + L(c x) (def of K + L) = K( x) + K( y) + L( x) + L( y) (K, L are linear) = (K L)( x) + (K L)( y).

(K + L)(c x) = K(c x) + L(c x) (def of K + L) = K( x) + K( y) + L( x) + L( y) (K, L are linear) = (K L)( x) + (K L)( y). Exercise 71 We have L( x) = x 1 L( v 1 ) + x 2 L( v 2 ) + + x n L( v n ) n = x i (a 1i w 1 + a 2i w 2 + + a mi w m ) i=1 ( n ) ( n ) ( n ) = x i a 1i w 1 + x i a 2i w 2 + + x i a mi w m i=1 Therefore y

More information

Normed & Inner Product Vector Spaces

Normed & Inner Product Vector Spaces Normed & Inner Product Vector Spaces ECE 174 Introduction to Linear & Nonlinear Optimization Ken Kreutz-Delgado ECE Department, UC San Diego Ken Kreutz-Delgado (UC San Diego) ECE 174 Fall 2016 1 / 27 Normed

More information

19. Principal Stresses

19. Principal Stresses 19. Principal Stresses I Main Topics A Cauchy s formula B Principal stresses (eigenvectors and eigenvalues) C Example 10/24/18 GG303 1 19. Principal Stresses hkp://hvo.wr.usgs.gov/kilauea/update/images.html

More information

4.4. Orthogonality. Note. This section is awesome! It is very geometric and shows that much of the geometry of R n holds in Hilbert spaces.

4.4. Orthogonality. Note. This section is awesome! It is very geometric and shows that much of the geometry of R n holds in Hilbert spaces. 4.4. Orthogonality 1 4.4. Orthogonality Note. This section is awesome! It is very geometric and shows that much of the geometry of R n holds in Hilbert spaces. Definition. Elements x and y of a Hilbert

More information