Vectors in Function Spaces

Size: px
Start display at page:

Download "Vectors in Function Spaces"

Transcription

1 Jim Lambers MAT 66 Spring Semester Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also known as a linear vector space, is a set of objects, called vectors, together with two operations: Addition of two vectors in V, which must be commutative, associative, and have an identity element, which is the zero vector. Each vector v must have an additive inverse v which, when added to v, yields the zero vector. Multiplication of a vector in V by a scalar, which is typically a real or complex number. The term scalar is used in this context, rather than number, because the multiplication process is scaling a given vector by a factor indicated by a given number. Scalar multiplication must satisfy distributive laws, and have an identity element, 1, such that 1v = v for any vector v V. Both operations must be closed, which means that the result of either operation must be a vector in V. That is, if u and v are two vectors in V, then u + v must also be in V, and αv must be in V for any scalar α. Example 1 The set of all points in n-dimensional space, R n, is a vector space. Addition is defined as follows: u 1 v 1 u 1 + v 1 u v u + v u + v = Scalar multiplication is defined by. u n +. v n αv = = αv 1 αv. αv n. = u + v. u n + v n Similarly, the set of all n-dimensional points whose coordinates are complex numbers, denoted by C n, is also a vector space.. In these next few examples, we introduce some vector spaces whose vectors are functions, which are also known as function spaces. Example The set of all polynomials of degree at most n, denoted by P n, is a vector space, in which addition and scalar multiplication are defined as follows. Given f(x), g(x) P n, (f + g)(x) = f(x) + g(x), (αf)(x) = αf(x). 1

2 These operations are closed, because adding two polynomials of degree at most n will not yield a sum whose degree is greater than n, and multiplying any polynomial by a nonzero scalar will not change its degree. Example 3 The set of all functions with power series of the form f(x) = a n x n, n= that are convergent on the interval ( 1, 1) is a vector space, in which addition and multiplication are defined as in the previous example. These operations are closed because the sum of two convergent series is also convergent, as is a scalar multiple of a convergent series. Example 4 The set of all continuous functions on the interval [a, b], denoted by C[a, b], is a vector space in which addition and scalar multiplication are defined as in the previous two examples. These operations are closed because the sum of two continuous functions, and a scalar multiple of a continuous function, is also continuous. A vector space V is most effectively described in terms of a set of specific vectors {v 1, v,...} that, in conjunction with the operations of addition and scalar multiplication, can be used to obtain every vector in the space. That is, for every vector v V, there must exist scalars c 1, c,..., such that v = c 1 v 1 + c v +. We say that v is a linear combination of v 1, v,..., and the scalars c 1, c,... are the coefficients of the linear combination. Ideally, it should be possible to express any vector v V as a unique linear combination of the vectors v 1, v,... that are to be used to describe all vectors in V. With this criteria in mind, we introduce the following two essential concepts from linear algebra: A set of vectors {v 1, v,..., v n } is linearly independent if the vector equation c 1 v 1 + c v + + a n v n = is satisfied if and only if c 1 = c = =. In other words, this set of vectors is linearly independent if it is not possible to express any vector in the set as a linear combination of other vectors in the set. This definition can be generalized in a natural way to an infinite set of vectors. If a set of vectors is not linearly independent, then we say that it is linearly dependent. A set of vectors {v 1, v,..., v n } spans a vector space V if, for any vector v V, there exist scalars c 1, c,..., a n such that v = c 1 v 1 + c v + + a n v n. That is, any vector in V can be expressed as a linear combination of vectors in the set. We define span{v 1, v,..., v n } to be the set of all linear combinations of v 1, v,..., v n. As with linear independence, the notion of span generalizes naturally to an infinite set of vectors.

3 We then say that a set of vectors {v 1, v,...} (which may be finite or infinite) is a basis for a vector space V if it is linearly independent, and if it spans V. This definition ensures that any vector in V is a unique linear combination of the vectors in the basis. If a basis for V is finite, then we say that V is finite-dimensional and define the dimension of V to be the number of elements in a basis; all bases of a finite-dimensional vector space must have the same number of elements. If V does not have a finite basis, then we say that V is infinite-dimensional. Example 5 The function space P 3, consisting of polynomials of degree at most 3, has a basis {1, x, x, x 3 }. It is clear that any polynomial in P 3 can be expressed as a linear combination of these basis functions, as the coefficients of any such polynomial are also the coefficients in the linear combination of these basis functions. To confirm linear independence, suppose that there exists constants c, c 1, c, and c 3 such that c (1) + c 1 x + c x + c 3 x 3 = for all x R. Then certainly this must be the case at x =, which requires that c 1 =. Substituting 3 other values of x into the above equation yields a system of 3 linear equations in the remaining 3 unknows c 1, c and c 3. It can be shown that the only solution of such a system of equations is the trivial solution c 1 = c = c 3 =. Therefore the set {1, x, x, x 3 } is linearly independent. An alternative basis consists of the first 4 Chebyshev polynomials {1, x, x 1, 4x 3 3x}. It can be confirmed using a similar approach that these polynomials are linearly independent. Example 6 The function space consisting of all power series that are convergent on the interval ( 1, 1) has as a basis the infinite set {1, x, x, x 3,...}. Using an inductive argument, it can be shown that this set is linearly independent Inner Product Recall that the dot product of two vectors u and v in R n is where u v = u 1 v 1 + u v + + u n v n = u v cos θ, u = u 1 + u + + u n is the magnitude or length of u, and θ is the angle between u and v, with θ radians. The dot product has the following properties: 1. u u = u. u (v + w) = u v + u w 3. u v = v u 4. u (cv) = c(u v) When u and v are perpendicular, then cos θ =. It follows that u v =, and we say that u and v are orthogonal. We would like to generalize the concept of a dot product to vectors in function spaces, and we also need to ensure that complex numbers are properly taken into account. To that end, we define the inner product of two functions f(x) and g(x) to be b a f(x)g(x)w(x) dx, 3

4 where w(x) is a weight function and, for any complex number z = x + iy, z = x iy is the complex conjugate of z. The interval of integration [a, b] depends on the function space under consideration. Using this definition, it can be verified that the inner product has the following properties: 1. f, g + h = f, g + f, h. g, f 3. f, cg = c f, g for any complex number c Note that the second property is slightly different from the corresponding property for vectors in R n, as it requires the complex conjugate. Combining the second and third property yields the result cf, g = c f, g. Inner Product Spaces and Hilbert Spaces Just as we use v to measure the magnitude of a vector v R n, we need a notion of magnitude for a function f(x) in a function space. To that end, we say that a function : V R is a norm on a vector space V if it satisfies the following conditions: 1. v for any vector v V, and v = if and only if v =.. αv = α v for any complex scalar α. 3. u + v u + v for any two vectors u, v V. This is known as the Triangle inequality. Any function that satisfies these conditions is useful for measuring the magnitude of a vector. Given an inner product,, we define v = v, v 1/ to be the norm induced by this inner product. Using the properties of the inner product, it can be shown that this norm satisfies the above conditions, except that for a function space, f may equal zero even if f is nonzero at isolated points, or more generally, if f is nonzero on a set of measure zero. We can now define a specific type of function space that will be of use to us. An inner product space is a vector space, together with an inner product and induced norm. A Hilbert space H is a function space, together with an inner product and induced norm, that is also complete. This means that there exists a basis ϕ 1, ϕ,... such that for any f H, there exists scalars c 1, c,... such that f = a n ϕ n. Example 7 As before, let P 3 be the space of polynomials of degree at most 3. If we use the inner product 1 f(s)g(s) ds, with weight function w(x) = 1, then P 3 is a Hilbert space. Let L (x) = 1 and L 1 (x) = x. Then L, L = L 1, L 1 = L, L 1 = ds = x ds = 1(x) ds =. 4 1 ds =, x dx = 3,

5 Here, we have used the fact that if f(x) is an even function, meaning that f( x) = f(x), then a a f(x) dx = a f(x) dx, whereas if f(x) is an odd function, meaning that f( x) = f(x), then a a f(x) dx =. Example 8 We denote by E(a, b) the inner product space consisting of all piecewise continuous functions on [a, b]. Its inner product is the standard inner product on (a, b), b a f(x)g(x) dx. More generally, we denote by E r (a, b) the inner product space of all piecewise continuous functions on [a, b], with inner product f, g r = b a f(x)g(x)r(x) dx, where r is a weight function. It must be continuous and positive on [a, b]. It will be seen later that E r (a, b), for such a weight function r, is actually a Hilbert space, as it is complete. Cauchy-Schwarz Inequality The Cauchy-Schwarz inequality, also known as the Schwarz inequality, states that if a norm on a vector space V is defined by f = f, f 1/ for any f V, where f, g is an inner product as defined previously, then f, g f g. We will prove this inequality in the case where V is a vector space defined over the real numbers; the proof can be generalized to a complex vector space. For f, g V and c R, with g, we have f cg, f cg. It follows from the properties of the inner product that f cg, f cg f, f f, cg cg, f + cg, cg f c f, g + c g. We now try to find the value of c that minimizes this expression. Differentiating with respect to c and equating to zero yields the equation f, g + c g =, 5

6 and therefore the minimum occurs when c = f, g / g. It follows that It follows that f c f, g + c g f f f f, g f, g f, g + g g 4 g f, g f, g + g g f, g g. f, g f g. Taking the square root of both sides yields the Cauchy-Schwarz inequality. Orthogonal Expansions Just as two vectors u, v R n are orthogonal if u v =, we say that two vectors u and v in an inner product space are orthogonal if u, v =. An orthogonal basis of an Hilbert space is particularly useful because the coefficients of a function with respect to such a basis are easily computed. Specifically, suppose a Hilbert space H has a basis {ϕ 1, ϕ,...} such that ϕ i, ϕ j = for i j. Then, let f H have the expansion f = a n ϕ n. If we take the inner product of both sides with ϕ k for some positive integer k, we obtain ϕ k, f = ϕ k, a n ϕ n = a n ϕ k, ϕ n = a k ϕ k, ϕ k, which yields the coefficients a n = ϕ n, f, n = 1,,.... ϕ n, ϕ n This is significant because it shows that the coefficients can be computed independently of one another. Recall that a vector u R n is a unit vector if u = 1, and that a unit vector can be obtained from a nonzero vector v by normalizing v, which means dividing by its magnitude: u = v/ v is a unit vector. Similarly, given a function f in a Hilbert space H, f is said to be normalized if f = 1. We then say that a set of function {ϕ 1, ϕ,...} is orthonormal if ϕ i, ϕ j = δ ij, where is called the Kronecker delta. δ ij = { i j 1 i = j 6

7 Example 9 Consider the infinite set of functions {sin nx}, with the inner product f(s)g(s) ds. Then we have, for positive integers m and n, with m n, sin mx, sin nx = sin mx sin nx dx = 1 cos[(m n)x] cos[(m + n)x] dx ( 1 1 = sin[(m n)x] sin[(m + n)x] m n m + n =, sin nx = sin nx, sin nx = = = x 1 n =. sin nx dx 1 cos nx dx sin nx Therefore, the functions in this set are orthogonal, but not orthonormal. Since the norm of each is /, it follows that the set { } sin nx is an orthonormal set. Then, to compute the coefficients {a n } in the expansion f(x) = a n sin nx, we need only compute the inner products a n = sin nx, f = This representation of f(x) is called a Fourier sine series. An expansion of a function f in an orthonormal basis {ϕ n }, f f(x) sin nx dx, n =, 1,,.... ˆf n ϕ n, ˆfn = ϕ n, f, n = 1,,..., is called a (generalized) Fourier expansion of f. The coefficients { ˆf n } are called the (generalized) Fourier coefficients of f. A truncated Fourier expansion ) f N = N ˆf n ϕ n 7

8 is called the orthogonal projection of f onto the subspace This orthogonal projection satisfies f f N = f V N = span{ϕ 1, ϕ,..., ϕ N }. N ˆf n f v, v V N. Taking the limit as N, and using the fact that f f N, we obtain Bessel s inequality ˆf n f. An immediate consequence of this inequality is the Riemann-Lebesgue Lemma lim ˆf n = lim ϕ n, f =. n n If the underlying function space is a Hilbert space, meaning that it is complete, then lim f f N =, N and we say that the Fourier expansion converges in the mean to f. properties of the norm and inner product that It then follows from the ˆf n = f. This is known as Parseval s identity. It is important to note that even if the Fourier expansion of f converges to f in the mean, this does not necessarily mean that the expansion agrees with f at every single point in the domain on which the inner product is defined. This is because, as mentioned earlier, that f, f = is possible even if f is nonzero at certain isolated points in the interval of integration. This will be illustrated in the following example. Example 1 Consider the function f(x) = x on (, ). We wish to obtain an expansion of this function in terms of the functions cos nx, n =, 1,,... and sin nx, n = 1,,..., which are known to be orthogonal with respect to the standard inner product on (, ), Therefore, our expansion will have the form f(x) = a + f(s)g(s) ds. (a n cos nx + b n sin nx). 8

9 Since f(x) is an odd function, a n = for n =, 1,,..., so we need only compute the coefficients {b n }, which are given by b n = = 1 sin nx, f sin nx, sin nx f(x) sin nx dx = 1 x sin nx dx = 1 [ 1n x cos nx + 1 n [ 1n x cos nx + 1n sin nx ] cos nx dx ] = 1 = cos n n = ( 1)n+1. n Therefore, the Fourier sine series for f(x) = x on (, ) is x = ( 1) n+1 sin nx. n Figure 1 shows how this series converges to f(x). Note that because the -periodic extension of f(x) has a discontinuity at x = k, where k is any odd integer, at these points the series converges to the average of the values of f(x) at these jumps, which is. The resulting oscillations near the discontinuity are an example of Gibbs phenomenon. Expansions and Inner Products Suppose that two functions f and g in a Hilbert space are expanded in the same orthonormal basis: f = ˆf n ϕ n, g = ĝ m ϕ m. m=1 Then, using the properties of the inner product, we have ˆf n ϕ n, ĝ m ϕ m = = = m=1 m=1 ˆf n ĝ n. m=1 ˆf n ĝ m ϕ n, ϕ m ˆf n ĝ m δ nm 9

10 Figure 1: Fourier sine series expansion of f(x) = x (red curve), compared to f(x) itself (blue curve) In other words, the inner product reduces to a dot product: f H g, where f and g are column vectors (which may be infinitely long) consisting of the coefficients { ˆf n } and {ĝ m }, respectively. For any vector v, we denote by v H the Hermitian transpose of v, which is the transpose and complex conjugate of v. In some texts, v H is written as v H or v. Note that in the case of f = g, we obtain which is again Parseval s identity. f = f, f = ˆf n, Example 11 The functions ϕ (x) = 1, ϕ n(x) = cos nx, n = 1,,... form an orthonormal set with respect to the inner product Now, consider the functions f(s)g(s) ds. ψ 1 (x) = cos 3 x + sin x + cos x + 1, ψ (x) = cos x cos x. 1

11 It can be verified directly by computing ϕ j, ψ k for j =, 1,, 3 and k = 1,, or by rewriting ψ 1 and ψ using trigonometric identities, that these functions have the following expansions in the basis {ϕ n } n= : ψ 1 (x) = 3 ϕ (x) ϕ 1(x) ϕ (x) + 4 ϕ 3(x), ψ (x) = ϕ (x) ϕ 1(x) + ϕ (x). Because the basis is orthonormal, we can compute ψ i, ψ j, for i, j = 1,, by computing dot products of the appropriate sets of coefficients: ( ) ( ) ( 3 7 ψ 1, ψ 1 = = 63 16, ψ 1, ψ = = 4, ( ) ψ 1, ψ = + = 7 8. ( ) ( ) + ) ( ) () 11

Hilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space.

Hilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space. Hilbert Spaces Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space. Vector Space. Vector space, ν, over the field of complex numbers,

More information

Vector Spaces. Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms.

Vector Spaces. Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms. Vector Spaces Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms. For each two vectors a, b ν there exists a summation procedure: a +

More information

Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold: Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v

More information

Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product

Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )

More information

Lecture 16: Bessel s Inequality, Parseval s Theorem, Energy convergence

Lecture 16: Bessel s Inequality, Parseval s Theorem, Energy convergence Introductory lecture notes on Partial Differential Equations - c Anthony Peirce. ot to be copied, used, or revised without explicit written permission from the copyright owner. ecture 6: Bessel s Inequality,

More information

Mathematics Department Stanford University Math 61CM/DM Inner products

Mathematics Department Stanford University Math 61CM/DM Inner products Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector

More information

The following definition is fundamental.

The following definition is fundamental. 1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic

More information

Linear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products

Linear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 6A: Inner products In this chapter, the field F = R or C. We regard F equipped with a conjugation χ : F F. If F =

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Hilbert Spaces. Contents

Hilbert Spaces. Contents Hilbert Spaces Contents 1 Introducing Hilbert Spaces 1 1.1 Basic definitions........................... 1 1.2 Results about norms and inner products.............. 3 1.3 Banach and Hilbert spaces......................

More information

Cambridge University Press The Mathematics of Signal Processing Steven B. Damelin and Willard Miller Excerpt More information

Cambridge University Press The Mathematics of Signal Processing Steven B. Damelin and Willard Miller Excerpt More information Introduction Consider a linear system y = Φx where Φ can be taken as an m n matrix acting on Euclidean space or more generally, a linear operator on a Hilbert space. We call the vector x a signal or input,

More information

Recall that any inner product space V has an associated norm defined by

Recall that any inner product space V has an associated norm defined by Hilbert Spaces Recall that any inner product space V has an associated norm defined by v = v v. Thus an inner product space can be viewed as a special kind of normed vector space. In particular every inner

More information

be the set of complex valued 2π-periodic functions f on R such that

be the set of complex valued 2π-periodic functions f on R such that . Fourier series. Definition.. Given a real number P, we say a complex valued function f on R is P -periodic if f(x + P ) f(x) for all x R. We let be the set of complex valued -periodic functions f on

More information

Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning:

Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning: Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u 2 1 + u 2 2 = u 2. Geometric Meaning: u v = u v cos θ. u θ v 1 Reason: The opposite side is given by u v. u v 2 =

More information

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces. Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

Outline of Fourier Series: Math 201B

Outline of Fourier Series: Math 201B Outline of Fourier Series: Math 201B February 24, 2011 1 Functions and convolutions 1.1 Periodic functions Periodic functions. Let = R/(2πZ) denote the circle, or onedimensional torus. A function f : C

More information

INNER PRODUCT SPACE. Definition 1

INNER PRODUCT SPACE. Definition 1 INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of

More information

CHAPTER VIII HILBERT SPACES

CHAPTER VIII HILBERT SPACES CHAPTER VIII HILBERT SPACES DEFINITION Let X and Y be two complex vector spaces. A map T : X Y is called a conjugate-linear transformation if it is a reallinear transformation from X into Y, and if T (λx)

More information

Math 276, Spring 2007 Additional Notes on Vectors

Math 276, Spring 2007 Additional Notes on Vectors Math 276, Spring 2007 Additional Notes on Vectors 1.1. Real Vectors. 1. Scalar Products If x = (x 1,..., x n ) is a vector in R n then the length of x is x = x 2 1 + + x2 n. We sometimes use the notation

More information

Mathematical Methods wk 1: Vectors

Mathematical Methods wk 1: Vectors Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm

More information

Mathematical Methods wk 1: Vectors

Mathematical Methods wk 1: Vectors Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm

More information

Fourier Series. 1. Review of Linear Algebra

Fourier Series. 1. Review of Linear Algebra Fourier Series In this section we give a short introduction to Fourier Analysis. If you are interested in Fourier analysis and would like to know more detail, I highly recommend the following book: Fourier

More information

Introduction to Signal Spaces

Introduction to Signal Spaces Introduction to Signal Spaces Selin Aviyente Department of Electrical and Computer Engineering Michigan State University January 12, 2010 Motivation Outline 1 Motivation 2 Vector Space 3 Inner Product

More information

General Inner Product and The Fourier Series

General Inner Product and The Fourier Series A Linear Algebra Approach Department of Mathematics University of Puget Sound 4-20-14 / Spring Semester Outline 1 2 Inner Product The inner product is an algebraic operation that takes two vectors and

More information

Methods of Mathematical Physics X1 Homework 3 Solutions

Methods of Mathematical Physics X1 Homework 3 Solutions Methods of Mathematical Physics - 556 X Homework 3 Solutions. (Problem 2.. from Keener.) Verify that l 2 is an inner product space. Specifically, show that if x, y l 2, then x, y x k y k is defined and

More information

MATH 220: INNER PRODUCT SPACES, SYMMETRIC OPERATORS, ORTHOGONALITY

MATH 220: INNER PRODUCT SPACES, SYMMETRIC OPERATORS, ORTHOGONALITY MATH 22: INNER PRODUCT SPACES, SYMMETRIC OPERATORS, ORTHOGONALITY When discussing separation of variables, we noted that at the last step we need to express the inhomogeneous initial or boundary data as

More information

14 Fourier analysis. Read: Boas Ch. 7.

14 Fourier analysis. Read: Boas Ch. 7. 14 Fourier analysis Read: Boas Ch. 7. 14.1 Function spaces A function can be thought of as an element of a kind of vector space. After all, a function f(x) is merely a set of numbers, one for each point

More information

2. Signal Space Concepts

2. Signal Space Concepts 2. Signal Space Concepts R.G. Gallager The signal-space viewpoint is one of the foundations of modern digital communications. Credit for popularizing this viewpoint is often given to the classic text of

More information

Functional Analysis Exercise Class

Functional Analysis Exercise Class Functional Analysis Exercise Class Week: December 4 8 Deadline to hand in the homework: your exercise class on week January 5. Exercises with solutions ) Let H, K be Hilbert spaces, and A : H K be a linear

More information

Section 7.5 Inner Product Spaces

Section 7.5 Inner Product Spaces Section 7.5 Inner Product Spaces With the dot product defined in Chapter 6, we were able to study the following properties of vectors in R n. ) Length or norm of a vector u. ( u = p u u ) 2) Distance of

More information

ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6]

ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] Inner products and Norms Inner product or dot product of 2 vectors u and v in R n : u.v = u 1 v 1 + u 2 v 2 + + u n v n Calculate u.v when u = 1 2 2 0 v = 1 0

More information

Topics in Fourier analysis - Lecture 2.

Topics in Fourier analysis - Lecture 2. Topics in Fourier analysis - Lecture 2. Akos Magyar 1 Infinite Fourier series. In this section we develop the basic theory of Fourier series of periodic functions of one variable, but only to the extent

More information

Ordinary Differential Equations II

Ordinary Differential Equations II Ordinary Differential Equations II February 23 2017 Separation of variables Wave eq. (PDE) 2 u t (t, x) = 2 u 2 c2 (t, x), x2 c > 0 constant. Describes small vibrations in a homogeneous string. u(t, x)

More information

2. Review of Linear Algebra

2. Review of Linear Algebra 2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

More information

Definition 1. A set V is a vector space over the scalar field F {R, C} iff. there are two operations defined on V, called vector addition

Definition 1. A set V is a vector space over the scalar field F {R, C} iff. there are two operations defined on V, called vector addition 6 Vector Spaces with Inned Product Basis and Dimension Section Objective(s): Vector Spaces and Subspaces Linear (In)dependence Basis and Dimension Inner Product 6 Vector Spaces and Subspaces Definition

More information

The Gram-Schmidt Process 1

The Gram-Schmidt Process 1 The Gram-Schmidt Process In this section all vector spaces will be subspaces of some R m. Definition.. Let S = {v...v n } R m. The set S is said to be orthogonal if v v j = whenever i j. If in addition

More information

HILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define

HILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define HILBERT SPACES AND THE RADON-NIKODYM THEOREM STEVEN P. LALLEY 1. DEFINITIONS Definition 1. A real inner product space is a real vector space V together with a symmetric, bilinear, positive-definite mapping,

More information

Inner Product Spaces

Inner Product Spaces Inner Product Spaces Introduction Recall in the lecture on vector spaces that geometric vectors (i.e. vectors in two and three-dimensional Cartesian space have the properties of addition, subtraction,

More information

MAT Linear Algebra Collection of sample exams

MAT Linear Algebra Collection of sample exams MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

More information

7: FOURIER SERIES STEVEN HEILMAN

7: FOURIER SERIES STEVEN HEILMAN 7: FOURIER SERIES STEVE HEILMA Contents 1. Review 1 2. Introduction 1 3. Periodic Functions 2 4. Inner Products on Periodic Functions 3 5. Trigonometric Polynomials 5 6. Periodic Convolutions 7 7. Fourier

More information

Inner product spaces. Layers of structure:

Inner product spaces. Layers of structure: Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty

More information

Jim Lambers MAT 169 Fall Semester Lecture 6 Notes. a n. n=1. S = lim s k = lim. n=1. n=1

Jim Lambers MAT 169 Fall Semester Lecture 6 Notes. a n. n=1. S = lim s k = lim. n=1. n=1 Jim Lambers MAT 69 Fall Semester 2009-0 Lecture 6 Notes These notes correspond to Section 8.3 in the text. The Integral Test Previously, we have defined the sum of a convergent infinite series to be the

More information

Numerical Linear Algebra

Numerical Linear Algebra University of Alabama at Birmingham Department of Mathematics Numerical Linear Algebra Lecture Notes for MA 660 (1997 2014) Dr Nikolai Chernov April 2014 Chapter 0 Review of Linear Algebra 0.1 Matrices

More information

LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

More information

Fourier and Wavelet Signal Processing

Fourier and Wavelet Signal Processing Ecole Polytechnique Federale de Lausanne (EPFL) Audio-Visual Communications Laboratory (LCAV) Fourier and Wavelet Signal Processing Martin Vetterli Amina Chebira, Ali Hormati Spring 2011 2/25/2011 1 Outline

More information

Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

Math Linear Algebra II. 1. Inner Products and Norms

Math Linear Algebra II. 1. Inner Products and Norms Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,

More information

MTH 2032 SemesterII

MTH 2032 SemesterII MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents

More information

Hilbert spaces. 1. Cauchy-Schwarz-Bunyakowsky inequality

Hilbert spaces. 1. Cauchy-Schwarz-Bunyakowsky inequality (October 29, 2016) Hilbert spaces Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ [This document is http://www.math.umn.edu/ garrett/m/fun/notes 2016-17/03 hsp.pdf] Hilbert spaces are

More information

Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India

Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India CHAPTER 9 BY Prof. M. Saha Professor of Mathematics The University of Burdwan West Bengal, India E-mail : mantusaha.bu@gmail.com Introduction and Objectives In the preceding chapters, we discussed normed

More information

3 Orthogonality and Fourier series

3 Orthogonality and Fourier series 3 Orthogonality and Fourier series We now turn to the concept of orthogonality which is a key concept in inner product spaces and Hilbert spaces. We start with some basic definitions. Definition 3.1. Let

More information

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 : Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =

More information

Math 3191 Applied Linear Algebra

Math 3191 Applied Linear Algebra Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have

More information

Quantum Computing Lecture 2. Review of Linear Algebra

Quantum Computing Lecture 2. Review of Linear Algebra Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces

More information

David Hilbert was old and partly deaf in the nineteen thirties. Yet being a diligent

David Hilbert was old and partly deaf in the nineteen thirties. Yet being a diligent Chapter 5 ddddd dddddd dddddddd ddddddd dddddddd ddddddd Hilbert Space The Euclidean norm is special among all norms defined in R n for being induced by the Euclidean inner product (the dot product). A

More information

Projection Theorem 1

Projection Theorem 1 Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality

More information

APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.

APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2. APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product

More information

GQE ALGEBRA PROBLEMS

GQE ALGEBRA PROBLEMS GQE ALGEBRA PROBLEMS JAKOB STREIPEL Contents. Eigenthings 2. Norms, Inner Products, Orthogonality, and Such 6 3. Determinants, Inverses, and Linear (In)dependence 4. (Invariant) Subspaces 3 Throughout

More information

6.1. Inner Product, Length and Orthogonality

6.1. Inner Product, Length and Orthogonality These are brief notes for the lecture on Friday November 13, and Monday November 1, 2009: they are not complete, but they are a guide to what I want to say on those days. They are guaranteed to be incorrect..1.

More information

1. General Vector Spaces

1. General Vector Spaces 1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

More information

Inner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that. (1) (2) (3) x x > 0 for x 0.

Inner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that. (1) (2) (3) x x > 0 for x 0. Inner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that (1) () () (4) x 1 + x y = x 1 y + x y y x = x y x αy = α x y x x > 0 for x 0 Consequently, (5) (6)

More information

MATH 5640: Fourier Series

MATH 5640: Fourier Series MATH 564: Fourier Series Hung Phan, UMass Lowell September, 8 Power Series A power series in the variable x is a series of the form a + a x + a x + = where the coefficients a, a,... are real or complex

More information

Contents. Appendix D (Inner Product Spaces) W-51. Index W-63

Contents. Appendix D (Inner Product Spaces) W-51. Index W-63 Contents Appendix D (Inner Product Spaces W-5 Index W-63 Inner city space W-49 W-5 Chapter : Appendix D Inner Product Spaces The inner product, taken of any two vectors in an arbitrary vector space, generalizes

More information

CONVERGENCE OF THE FOURIER SERIES

CONVERGENCE OF THE FOURIER SERIES CONVERGENCE OF THE FOURIER SERIES SHAW HAGIWARA Abstract. The Fourier series is a expression of a periodic, integrable function as a sum of a basis of trigonometric polynomials. In the following, we first

More information

Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.

Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space. Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space

More information

Chapter 2. Vectors and Vector Spaces

Chapter 2. Vectors and Vector Spaces 2.1. Operations on Vectors 1 Chapter 2. Vectors and Vector Spaces Section 2.1. Operations on Vectors Note. In this section, we define several arithmetic operations on vectors (especially, vector addition

More information

MATH 115A: SAMPLE FINAL SOLUTIONS

MATH 115A: SAMPLE FINAL SOLUTIONS MATH A: SAMPLE FINAL SOLUTIONS JOE HUGHES. Let V be the set of all functions f : R R such that f( x) = f(x) for all x R. Show that V is a vector space over R under the usual addition and scalar multiplication

More information

Linear Algebra. Session 12

Linear Algebra. Session 12 Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

More information

Functions of Several Variables

Functions of Several Variables Jim Lambers MAT 419/519 Summer Session 2011-12 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Functions of Several Variables We now generalize the results from the previous section,

More information

08a. Operators on Hilbert spaces. 1. Boundedness, continuity, operator norms

08a. Operators on Hilbert spaces. 1. Boundedness, continuity, operator norms (February 24, 2017) 08a. Operators on Hilbert spaces Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ [This document is http://www.math.umn.edu/ garrett/m/real/notes 2016-17/08a-ops

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,

More information

Real Variables # 10 : Hilbert Spaces II

Real Variables # 10 : Hilbert Spaces II randon ehring Real Variables # 0 : Hilbert Spaces II Exercise 20 For any sequence {f n } in H with f n = for all n, there exists f H and a subsequence {f nk } such that for all g H, one has lim (f n k,

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

where the bar indicates complex conjugation. Note that this implies that, from Property 2, x,αy = α x,y, x,y X, α C.

where the bar indicates complex conjugation. Note that this implies that, from Property 2, x,αy = α x,y, x,y X, α C. Lecture 4 Inner product spaces Of course, you are familiar with the idea of inner product spaces at least finite-dimensional ones. Let X be an abstract vector space with an inner product, denoted as,,

More information

Physics 342 Lecture 2. Linear Algebra I. Lecture 2. Physics 342 Quantum Mechanics I

Physics 342 Lecture 2. Linear Algebra I. Lecture 2. Physics 342 Quantum Mechanics I Physics 342 Lecture 2 Linear Algebra I Lecture 2 Physics 342 Quantum Mechanics I Wednesday, January 27th, 21 From separation of variables, we move to linear algebra Roughly speaking, this is the study

More information

Math 489AB A Very Brief Intro to Fourier Series Fall 2008

Math 489AB A Very Brief Intro to Fourier Series Fall 2008 Math 489AB A Very Brief Intro to Fourier Series Fall 8 Contents Fourier Series. The coefficients........................................ Convergence......................................... 4.3 Convergence

More information

Orthonormal Systems. Fourier Series

Orthonormal Systems. Fourier Series Yuliya Gorb Orthonormal Systems. Fourier Series October 31 November 3, 2017 Yuliya Gorb Orthonormal Systems (cont.) Let {e α} α A be an orthonormal set of points in an inner product space X. Then {e α}

More information

(v, w) = arccos( < v, w >

(v, w) = arccos( < v, w > MA322 F all203 Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: For all v,

More information

Linear Analysis Lecture 5

Linear Analysis Lecture 5 Linear Analysis Lecture 5 Inner Products and V Let dim V < with inner product,. Choose a basis B and let v, w V have coordinates in F n given by x 1. x n and y 1. y n, respectively. Let A F n n be the

More information

The Transpose of a Vector

The Transpose of a Vector 8 CHAPTER Vectors The Transpose of a Vector We now consider the transpose of a vector in R n, which is a row vector. For a vector u 1 u. u n the transpose is denoted by u T = [ u 1 u u n ] EXAMPLE -5 Find

More information

Advanced Computational Fluid Dynamics AA215A Lecture 2 Approximation Theory. Antony Jameson

Advanced Computational Fluid Dynamics AA215A Lecture 2 Approximation Theory. Antony Jameson Advanced Computational Fluid Dynamics AA5A Lecture Approximation Theory Antony Jameson Winter Quarter, 6, Stanford, CA Last revised on January 7, 6 Contents Approximation Theory. Least Squares Approximation

More information

Vector Calculus. Lecture Notes

Vector Calculus. Lecture Notes Vector Calculus Lecture Notes Adolfo J. Rumbos c Draft date November 23, 211 2 Contents 1 Motivation for the course 5 2 Euclidean Space 7 2.1 Definition of n Dimensional Euclidean Space........... 7 2.2

More information

Fall f(x)g(x) dx. The starting place for the theory of Fourier series is that the family of functions {e inx } n= is orthonormal, that is

Fall f(x)g(x) dx. The starting place for the theory of Fourier series is that the family of functions {e inx } n= is orthonormal, that is 18.103 Fall 2013 1. Fourier Series, Part 1. We will consider several function spaces during our study of Fourier series. When we talk about L p ((, π)), it will be convenient to include the factor 1/ in

More information

There are two things that are particularly nice about the first basis

There are two things that are particularly nice about the first basis Orthogonality and the Gram-Schmidt Process In Chapter 4, we spent a great deal of time studying the problem of finding a basis for a vector space We know that a basis for a vector space can potentially

More information

which arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i

which arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i MODULE 6 Topics: Gram-Schmidt orthogonalization process We begin by observing that if the vectors {x j } N are mutually orthogonal in an inner product space V then they are necessarily linearly independent.

More information

Page 52. Lecture 3: Inner Product Spaces Dual Spaces, Dirac Notation, and Adjoints Date Revised: 2008/10/03 Date Given: 2008/10/03

Page 52. Lecture 3: Inner Product Spaces Dual Spaces, Dirac Notation, and Adjoints Date Revised: 2008/10/03 Date Given: 2008/10/03 Page 5 Lecture : Inner Product Spaces Dual Spaces, Dirac Notation, and Adjoints Date Revised: 008/10/0 Date Given: 008/10/0 Inner Product Spaces: Definitions Section. Mathematical Preliminaries: Inner

More information

Lecture Notes for Inf-Mat 3350/4350, Tom Lyche

Lecture Notes for Inf-Mat 3350/4350, Tom Lyche Lecture Notes for Inf-Mat 3350/4350, 2007 Tom Lyche August 5, 2007 2 Contents Preface vii I A Review of Linear Algebra 1 1 Introduction 3 1.1 Notation............................... 3 2 Vectors 5 2.1 Vector

More information

SECTION v 2 x + v 2 y, (5.1)

SECTION v 2 x + v 2 y, (5.1) CHAPTER 5 5.1 Normed Spaces SECTION 5.1 171 REAL AND COMPLEX NORMED, METRIC, AND INNER PRODUCT SPACES So far, our studies have concentrated only on properties of vector spaces that follow from Definition

More information

Algebra II. Paulius Drungilas and Jonas Jankauskas

Algebra II. Paulius Drungilas and Jonas Jankauskas Algebra II Paulius Drungilas and Jonas Jankauskas Contents 1. Quadratic forms 3 What is quadratic form? 3 Change of variables. 3 Equivalence of quadratic forms. 4 Canonical form. 4 Normal form. 7 Positive

More information

(v, w) = arccos( < v, w >

(v, w) = arccos( < v, w > MA322 F all206 Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: Commutativity:

More information

(v, w) = arccos( < v, w >

(v, w) = arccos( < v, w > MA322 Sathaye Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: For all v

More information

1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,

1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e., Abstract Hilbert Space Results We have learned a little about the Hilbert spaces L U and and we have at least defined H 1 U and the scale of Hilbert spaces H p U. Now we are going to develop additional

More information

Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space

Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) Contents 1 Vector Spaces 1 1.1 The Formal Denition of a Vector Space.................................. 1 1.2 Subspaces...................................................

More information

Exercise Solutions to Functional Analysis

Exercise Solutions to Functional Analysis Exercise Solutions to Functional Analysis Note: References refer to M. Schechter, Principles of Functional Analysis Exersize that. Let φ,..., φ n be an orthonormal set in a Hilbert space H. Show n f n

More information

Linear Algebra in Hilbert Space

Linear Algebra in Hilbert Space Physics 342 Lecture 16 Linear Algebra in Hilbert Space Lecture 16 Physics 342 Quantum Mechanics I Monday, March 1st, 2010 We have seen the importance of the plane wave solutions to the potentialfree Schrödinger

More information

Fourier Series. Spectral Analysis of Periodic Signals

Fourier Series. Spectral Analysis of Periodic Signals Fourier Series. Spectral Analysis of Periodic Signals he response of continuous-time linear invariant systems to the complex exponential with unitary magnitude response of a continuous-time LI system at

More information

Definitions for Quizzes

Definitions for Quizzes Definitions for Quizzes Italicized text (or something close to it) will be given to you. Plain text is (an example of) what you should write as a definition. [Bracketed text will not be given, nor does

More information

Chapter 6: Orthogonality

Chapter 6: Orthogonality Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products

More information