Section 7.5 Inner Product Spaces
|
|
- Kristian Booth
- 5 years ago
- Views:
Transcription
1 Section 7.5 Inner Product Spaces With the dot product defined in Chapter 6, we were able to study the following properties of vectors in R n. ) Length or norm of a vector u. ( u = p u u ) 2) Distance of two given vectors u and v. ( u - v ) 3) Orthogonality of two vectors u and v. (whether u v = 0) Now, how do we describe the same properties of vectors in other types of vector spaces? For example, ) How do we define the norm of the function f(x) in C([a, b])? 2) How do we determine whether the polynomials f(x) and g(x) in P 2 are orthogonal? 3) How do we calculate the distance between the two matrices A and B in M 4x3?
2 Section 7.5 Inner Product Spaces Definition. Property: Many inner products may be defined on a vector space. Proof If, is an inner product, then, r defined by u, v r = r u, v is also an inner product for any r > 0. Definition. 2
3 Example: The dot product is an inner product on R n. Definition. 3
4 Example: V = C([a, b]) = {f f : [a, b] R, f is continuous} is a vector space, and the function, : V V R defined by f, g = f ( t) g( t) dt a f, g V is an inner product on V. Axiom : f 2 is continuous and non-negative. f 0 f 2 (t 0 ) > 0 for some t 0 [a, b]. f 2 (t) > p > 0 [t 0 -r/2, t 0 +r/2] [a, b]. Axioms 2-4: You examine them. b b 2 f, f = f ( t) dt r p > a 0. 4
5 Example: A, B = trace(ab T ) is the Frobenius inner product on R n n. Axiom : A, A = trace(aa T ) = Σ i,j n (a ij ) 2 > 0 A O. Axiom 2: trace(ab T ) = trace(ab T ) T = trace(ba T ). Axioms 3-4: You examine them. 5
6 Definition. For any vector v in an inner product space V,thenorm or length of v is denoted and defined as v = hv, vi /2. The distance between u, v 2 V is defined as u v. Example: The Frobenius norm in R n n is A = [Σ i,j n (a ij ) 2 ] /2, since A, B = trace(ab T ) = Σ i,j n a ij b ij. Property: Inner products and norms satisfy the elementary properties stated in Theorem 6., the Cauchy-Schwarz inequality, and the triangle inequality in Section 6.. Proof You show it. 6
7 Theorem 6. Let u and v be vectors in R n and c be a scalar in R. (a) u u = u 2. (b) u u = 0 if and only if u = 0. (c) u v = v u. (d) u (v + w) =u v + u w. (e) (v + w) u = v u + w u. (f) (cu) v = c(u u) =u (cu). (g) cu = c u. Let V be an inner product space and let u and v be vectors in V and c be a scalar in R. It can be shown that (a) hu, ui = u 2. (b) hu, ui = 0 if and only if u = 0. (c) hu, vi = hv, ui. (d) hu, v + wi = hu, vi + hu, wi. (e) hv + w, ui = hv, ui + hw, ui. (f) hcu, vi = c hu, ui = hu,cui. (g) cu = c u. 7
8 Theorem 6.2 (Pythagorean theorem in R n ) Let u and v be vectors in R n.thenu and v are orthogonal if and only if u + v 2 = u 2 + v 2. Proof u + v 2 = u 2 +2u v + v 2 = 0 if and only if u and v are orthogonal Pythagorean theorem in any inner product space V Let V be an inner product space and let u and v be vectors in V. Then u and v are orthogonal if and only if u + v 2 = u 2 + v 2. 8 Proof u + v 2 = u 2 +2hu, vi + v 2 = 0 if and only if u and v are orthogonal
9 Theorem 6.3 (Cauchy-Schwarz inequality) For any vectors u and v in R n,wehave Proof Using Theorem 6.2. u v apple u v. Cauchy-Schwarz inequality in any inner product space V Let V be an inner product space. For any vectors u and v in V,wehave hu, vi apple u v. Proof Using Pythagorean Theorem. Example: A Cauchy-Schwarz inequality in C([a, b]) Z b a 9 f(t)g(t)dt! 2 apple Z b a! Z! b f 2 (t)dt g 2 (t)dt a
10 Theorem 6.4 (Triangle inequality) For any vectors u and v in R n,wehave u + v apple u + v. Proof Using Theorem 6.3. Triangle inequality in any inner product space V Let V be an inner product space. For any vectors u and v in V,wehave u + v apple u + v. Proof Using Cauchy-Schwartz inequality. 0
11 Definition. In an inner product space V, the vectors u, v are called orthogonal if hu, vi = 0, a vector u is called a unit vector if u =, a subset S is called orthogonal if hu, vi = 0 for all distinct u, v 2 S, and S is called orthonormal if S is orthogonal and u = for all u 2 S. Properties:. Every nonzero vector v in an inner product space may be changed into a unit normalized vector (/ v )v, and every orthogonal subset with only nonzero vectors may be changed into an orthonormal subset without affecting the subspace spanned. 2. An orthogonal set of nonzero vectors is L.I., no matter the set is finite or infinite. Example: In the inner product space C([0, 2π]), the vectors f (t) = sin 3t and g(t) = cos 2t are orthogonal, since
12 Example: In the vector space of trigonometric polynomials T [0, 2 ] = Span {, cos t, sin t, cos 2t, sin 2t,, cos nt, sin nt, } = Span S S is orthogonal, since hcos nt, sin mti = hcos nt, cos mti = hsin nt, sin mti = Z 2 Z0 2 Z cos nt sin mtdt =0, 8n, m 0 cos nt cos mtdt =0, 8n 6= m sin nt sin mtdt =0, 8n 6= m S is a basis of T [0, 2π]. 2
13 Theorem 6.5 (Gram-Schmidt Process) Let {u, u 2,, u k } be a basis for a subspace W of R n.define v = u, v 2 = u 2 u 2 v v 2 v,. v k = u k u k v v 2 v u k v 2 v 2 2 v 2 u k v k v k 2 v k. Then {v, v 2,, v i } is an orthogonal set of nonzero vectors such that Span {v, v 2,, v i } = Span {u, u 2,, u i } for each i. So{v, v 2,, v k } is an orthogonal basis for W. 3
14 Gram-Schmidt Process for any inner product space Let {u, u 2,, u k } be a basis for an inner product space V.Define v = u, v 2 = u 2 hu 2, v i v 2 v,. v k = u k hu k, v i v 2 v hu k, v 2 i v 2 v 2 hu k, v k i v 2 v k. Then {v, v 2,, v i } is an orthogonal set of nonzero vectors such that Span {v, v 2,, v i } = Span {u, u 2,, u i } for each i. So{v, v 2,, v k } is an orthogonal basis for W. Proposition: The Gram-Schmidt process is valid for any inner product space. Proof You show it. Corollary: Every finite-dimensional inner product space has an orthonormal basis. 4
15 Example: P 2 is an inner product space with the following inner product f, g = f ( x) g( x) dx f, g P 2. From a basis B = {, x, x2 } = {u, u 2, u 3 } of P 2, an orthogonal basis {v, v 2, v 3 } may be obtained by applying the Gram-Schmidt process. 5
16 Example: P 2 is an inner product space with the following inner product f, g = f ( x) g( x) dx f, g P 2. From a basis B = {, x, x2 } = {u, u 2, u 3 } of P 2, an orthogonal basis {v, v 2, v 3 } may be obtained by applying the Gram-Schmidt process. v = u = v 2 = u 2 hu 2, v i v 2 v = x R R t dt () = x 0 =x 2 dt v 3 = u 3 hu 3, v i v 2 v hu 3, v 2 i v 2 2 v 2 = x 2 R t2 dt R 2 dt () R t2 tdt R t2 dt (x) = x x = x2 3. 6
17 To get an orthonormal basis, compute v = s Z 2 dx = p 2 v 2 = s Z x 2 dx = r 2 3 and get v v, v 3 = v 2 v 2, s Z 2 x 2 dx = 3 v 3 v 3, = r 8 45 For P with the same inner product and the basis B = {, x, x2, }, the same procedure may be applied to obtain an orthonormal basis {p 0 (x), p (x), p 2 (x), }, called the normalized Legendre polynomials. Note that p 0 (x), p (x), and p 2 (x) are the above three orthonormal ones. ( p 2, r r x, 8 ) x 2 3 7
18 Proposition Suppose that V is an inner product space and at W is a finite-dimensional subspace of V. For every v in V,thereexistuniquew 2 W and z 2 W? such that v = w + z. The vector v is called the orthogonal projection of v onto W, and we have w = hv, v i v + hv, v 2 i v hv, v n i v n if {v, v 2,, v n } is an orthonormal basis of W. Proof You show that the proof of Theorem 6.7 in Section 6.3 is also applicable here. Corollary: Under the notations in the above Proposition, among all vectors in W, the vectors closest to v is w. Proof Follow the derivations of the closest vector property in Section 6.3. Since the closeness is measured by the distance, which involves the sum (integral) of a square of the difference vector (function), the closest vector is called the least-square approximation. 8
19 Example: P 2 with the inner product f, g = f ( x) g( x) dx for all f, g P 2 is a finite-dimensional subspace of C([-, ]). 9 To v = f (x) = C([-, ]), the least-squares approxi- mation by a polynomial with degree 2 is the orthogonal projection of f onto P 2. odd function and get 3p x Thus take the orthonormal basis {v, rv 2, v 3 }, where 45 v = p 2 v 2 = 0 r 3 2 x w = hv, v i v + hv, v 2 i v 2 + hv, v 3 i v 3 = = 9 7 x Z 3p x r 3 2 xdx! r 3 2 x v 3 = 8 0 x 2 3 even function
20 Definition. The least-squares approximation by the trigonometric polynomials of a continuous periodic function f (t) of period 2π. Periodicity consider the approximation over a period [0, 2π]. f ( ) C([0, 2π]), and can be orthogonally projected onto a subspace W n spanned by an orthogonal set S n = {, cos t, sin t, cos 2t, sin 2t,,, cos nt, sin nt} To have an orthonormal basis for W n, compute s Z 2 20 = cos kt = 0 dt = p 2 s Z 2 sin kt = 0 cos 2 ktdt = s 2 Z 2 0 ( + cos 2kt)dt =
21 and get the orthonormal basis B n = p 2, cos t, sin t,, cos 2t, sin 2t,, cos nt, sin nt Suppose that f ( ) is the sawtooth function π f(t) = 2 t if 0 apple t apple t 3 if apple t apple 2 2 2
22 Let f n be the least-squares approximation of f by W n (the orthogonal projection of f onto W n ). Then f n = f, p 2 p 2 + f, cos t p cos t + f, sin t sin t + + f, cos nt p cos nt + f, sin nt sin nt Now, f, 22 f, p = Z p 2 2 cos kt = and f, = 0 Z 0 4 k 2 ( ( )k ) sin kt =0 2 t dt + p Z t cos kt dt + p Z 2 2 t 3 dt =0+0 2 t 3 cos kt dt
23 f n (t) = 8 2 apple cos t 2 + cos 3t cos nt n 2 23
24 24 Homework Set for Section 7.5 Section 7.5: Problems, 4, 9, 3, 7, 45, 46, 5, 53, 60, 62, 63, 64, 7, 75
Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:
Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationwhich arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i
MODULE 6 Topics: Gram-Schmidt orthogonalization process We begin by observing that if the vectors {x j } N are mutually orthogonal in an inner product space V then they are necessarily linearly independent.
More informationIntroduction to Signal Spaces
Introduction to Signal Spaces Selin Aviyente Department of Electrical and Computer Engineering Michigan State University January 12, 2010 Motivation Outline 1 Motivation 2 Vector Space 3 Inner Product
More informationMTH 309Y 37. Inner product spaces. = a 1 b 1 + a 2 b a n b n
MTH 39Y 37. Inner product spaces Recall: ) The dot product in R n : a. a n b. b n = a b + a 2 b 2 +...a n b n 2) Properties of the dot product: a) u v = v u b) (u + v) w = u w + v w c) (cu) v = c(u v)
More informationVector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.
Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar
More informationInner Product Spaces
Inner Product Spaces Introduction Recall in the lecture on vector spaces that geometric vectors (i.e. vectors in two and three-dimensional Cartesian space have the properties of addition, subtraction,
More informationMath 24 Spring 2012 Sample Homework Solutions Week 8
Math 4 Spring Sample Homework Solutions Week 8 Section 5. (.) Test A M (R) for diagonalizability, and if possible find an invertible matrix Q and a diagonal matrix D such that Q AQ = D. ( ) 4 (c) A =.
More informationINNER PRODUCT SPACE. Definition 1
INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of
More informationRecall that any inner product space V has an associated norm defined by
Hilbert Spaces Recall that any inner product space V has an associated norm defined by v = v v. Thus an inner product space can be viewed as a special kind of normed vector space. In particular every inner
More informationLINEAR ALGEBRA W W L CHEN
LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,
More informationLinear Algebra. Session 12
Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)
More informationFinite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product
Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )
More informationj=1 [We will show that the triangle inequality holds for each p-norm in Chapter 3 Section 6.] The 1-norm is A F = tr(a H A).
Math 344 Lecture #19 3.5 Normed Linear Spaces Definition 3.5.1. A seminorm on a vector space V over F is a map : V R that for all x, y V and for all α F satisfies (i) x 0 (positivity), (ii) αx = α x (scale
More informationVectors in Function Spaces
Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More informationMath 413/513 Chapter 6 (from Friedberg, Insel, & Spence)
Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) David Glickenstein December 7, 2015 1 Inner product spaces In this chapter, we will only consider the elds R and C. De nition 1 Let V be a vector
More informationCHAPTER II HILBERT SPACES
CHAPTER II HILBERT SPACES 2.1 Geometry of Hilbert Spaces Definition 2.1.1. Let X be a complex linear space. An inner product on X is a function, : X X C which satisfies the following axioms : 1. y, x =
More informationThere are two things that are particularly nice about the first basis
Orthogonality and the Gram-Schmidt Process In Chapter 4, we spent a great deal of time studying the problem of finding a basis for a vector space We know that a basis for a vector space can potentially
More informationRecall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning:
Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u 2 1 + u 2 2 = u 2. Geometric Meaning: u v = u v cos θ. u θ v 1 Reason: The opposite side is given by u v. u v 2 =
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week: December 4 8 Deadline to hand in the homework: your exercise class on week January 5. Exercises with solutions ) Let H, K be Hilbert spaces, and A : H K be a linear
More informationy 2 . = x 1y 1 + x 2 y x + + x n y n 2 7 = 1(2) + 3(7) 5(4) = 3. x x = x x x2 n.
6.. Length, Angle, and Orthogonality In this section, we discuss the defintion of length and angle for vectors and define what it means for two vectors to be orthogonal. Then, we see that linear systems
More informationorthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis,
5 Orthogonality Goals: We use scalar products to find the length of a vector, the angle between 2 vectors, projections, orthogonal relations between vectors and subspaces Then we study some applications
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationMathematics Department Stanford University Math 61CM/DM Inner products
Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector
More informationMath Real Analysis II
Math 4 - Real Analysis II Solutions to Homework due May Recall that a function f is called even if f( x) = f(x) and called odd if f( x) = f(x) for all x. We saw that these classes of functions had a particularly
More informationMATH 5640: Fourier Series
MATH 564: Fourier Series Hung Phan, UMass Lowell September, 8 Power Series A power series in the variable x is a series of the form a + a x + a x + = where the coefficients a, a,... are real or complex
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationFourier and Wavelet Signal Processing
Ecole Polytechnique Federale de Lausanne (EPFL) Audio-Visual Communications Laboratory (LCAV) Fourier and Wavelet Signal Processing Martin Vetterli Amina Chebira, Ali Hormati Spring 2011 2/25/2011 1 Outline
More informationLecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.
Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an
More informationFunctional Analysis Review
Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all
More informationInner Product Spaces 6.1 Length and Dot Product in R n
Inner Product Spaces 6.1 Length and Dot Product in R n Summer 2017 Goals We imitate the concept of length and angle between two vectors in R 2, R 3 to define the same in the n space R n. Main topics are:
More informationSolutions for MAS277 Problems
Solutions for MAS77 Problems Solutions for Chapter problems: Inner product spaces 1. No. The only part that fails, however, is the condition that f, f should mean that f. If f is a non-zero function that
More informationProjection Theorem 1
Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality
More informationLinear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products
Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 6A: Inner products In this chapter, the field F = R or C. We regard F equipped with a conjugation χ : F F. If F =
More information1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det
What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix
More informationFunctional Analysis. James Emery. Edit: 8/7/15
Functional Analysis James Emery Edit: 8/7/15 Contents 0.1 Green s functions in Ordinary Differential Equations...... 2 0.2 Integral Equations........................ 2 0.2.1 Fredholm Equations...................
More informationMATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.
MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v
More information(v, w) = arccos( < v, w >
MA322 Sathaye Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: For all v
More informationMathematical Methods wk 1: Vectors
Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm
More informationMathematical Methods wk 1: Vectors
Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm
More informationBest approximation in the 2-norm
Best approximation in the 2-norm Department of Mathematical Sciences, NTNU september 26th 2012 Vector space A real vector space V is a set with a 0 element and three operations: Addition: x, y V then x
More informationFunctional Analysis Review
Functional Analysis Review Lorenzo Rosasco slides courtesy of Andre Wibisono 9.520: Statistical Learning Theory and Applications September 9, 2013 1 2 3 4 Vector Space A vector space is a set V with binary
More informationv = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :
Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =
More informationApplied Analysis (APPM 5440): Final exam 1:30pm 4:00pm, Dec. 14, Closed books.
Applied Analysis APPM 44: Final exam 1:3pm 4:pm, Dec. 14, 29. Closed books. Problem 1: 2p Set I = [, 1]. Prove that there is a continuous function u on I such that 1 ux 1 x sin ut 2 dt = cosx, x I. Define
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationLINEAR ALGEBRA QUESTION BANK
LINEAR ALGEBRA QUESTION BANK () ( points total) Circle True or False: TRUE / FALSE: If A is any n n matrix, and I n is the n n identity matrix, then I n A = AI n = A. TRUE / FALSE: If A, B are n n matrices,
More information(v, w) = arccos( < v, w >
MA322 F all203 Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: For all v,
More informationORTHOGONALITY AND LEAST-SQUARES [CHAP. 6]
ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] Inner products and Norms Inner product or dot product of 2 vectors u and v in R n : u.v = u 1 v 1 + u 2 v 2 + + u n v n Calculate u.v when u = 1 2 2 0 v = 1 0
More informationMath 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.
Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,
More informationMath 261 Lecture Notes: Sections 6.1, 6.2, 6.3 and 6.4 Orthogonal Sets and Projections
Math 6 Lecture Notes: Sections 6., 6., 6. and 6. Orthogonal Sets and Projections We will not cover general inner product spaces. We will, however, focus on a particular inner product space the inner product
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationHILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define
HILBERT SPACES AND THE RADON-NIKODYM THEOREM STEVEN P. LALLEY 1. DEFINITIONS Definition 1. A real inner product space is a real vector space V together with a symmetric, bilinear, positive-definite mapping,
More information(v, w) = arccos( < v, w >
MA322 F all206 Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: Commutativity:
More informationMTH 2310, FALL Introduction
MTH 2310, FALL 2011 SECTION 6.2: ORTHOGONAL SETS Homework Problems: 1, 5, 9, 13, 17, 21, 23 1, 27, 29, 35 1. Introduction We have discussed previously the benefits of having a set of vectors that is linearly
More informationInner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that. (1) (2) (3) x x > 0 for x 0.
Inner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that (1) () () (4) x 1 + x y = x 1 y + x y y x = x y x αy = α x y x x > 0 for x 0 Consequently, (5) (6)
More informationAlgebra II. Paulius Drungilas and Jonas Jankauskas
Algebra II Paulius Drungilas and Jonas Jankauskas Contents 1. Quadratic forms 3 What is quadratic form? 3 Change of variables. 3 Equivalence of quadratic forms. 4 Canonical form. 4 Normal form. 7 Positive
More informationReproducing Kernel Hilbert Spaces Class 03, 15 February 2006 Andrea Caponnetto
Reproducing Kernel Hilbert Spaces 9.520 Class 03, 15 February 2006 Andrea Caponnetto About this class Goal To introduce a particularly useful family of hypothesis spaces called Reproducing Kernel Hilbert
More informationMATH 235: Inner Product Spaces, Assignment 7
MATH 235: Inner Product Spaces, Assignment 7 Hand in questions 3,4,5,6,9, by 9:3 am on Wednesday March 26, 28. Contents Orthogonal Basis for Inner Product Space 2 2 Inner-Product Function Space 2 3 Weighted
More informationChapter 6 Inner product spaces
Chapter 6 Inner product spaces 6.1 Inner products and norms Definition 1 Let V be a vector space over F. An inner product on V is a function, : V V F such that the following conditions hold. x+z,y = x,y
More informationHilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space.
Hilbert Spaces Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space. Vector Space. Vector space, ν, over the field of complex numbers,
More informationVector Spaces. Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms.
Vector Spaces Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms. For each two vectors a, b ν there exists a summation procedure: a +
More informationTopics in Fourier analysis - Lecture 2.
Topics in Fourier analysis - Lecture 2. Akos Magyar 1 Infinite Fourier series. In this section we develop the basic theory of Fourier series of periodic functions of one variable, but only to the extent
More informationHilbert Spaces: Infinite-Dimensional Vector Spaces
Hilbert Spaces: Infinite-Dimensional Vector Spaces PHYS 500 - Southern Illinois University October 27, 2016 PHYS 500 - Southern Illinois University Hilbert Spaces: Infinite-Dimensional Vector Spaces October
More informationMODULE 8 Topics: Null space, range, column space, row space and rank of a matrix
MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix Definition: Let L : V 1 V 2 be a linear operator. The null space N (L) of L is the subspace of V 1 defined by N (L) = {x
More informationMath Linear Algebra II. 1. Inner Products and Norms
Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,
More informationHilbert Spaces. Contents
Hilbert Spaces Contents 1 Introducing Hilbert Spaces 1 1.1 Basic definitions........................... 1 1.2 Results about norms and inner products.............. 3 1.3 Banach and Hilbert spaces......................
More informationIntroduction to Linear Algebra, Second Edition, Serge Lange
Introduction to Linear Algebra, Second Edition, Serge Lange Chapter I: Vectors R n defined. Addition and scalar multiplication in R n. Two geometric interpretations for a vector: point and displacement.
More informationHomework 11 Solutions. Math 110, Fall 2013.
Homework 11 Solutions Math 110, Fall 2013 1 a) Suppose that T were self-adjoint Then, the Spectral Theorem tells us that there would exist an orthonormal basis of P 2 (R), (p 1, p 2, p 3 ), consisting
More informationMath 1180, Notes, 14 1 C. v 1 v n v 2. C A ; w n. A and w = v i w i : v w = i=1
Math 8, 9 Notes, 4 Orthogonality We now start using the dot product a lot. v v = v v n then by Recall that if w w ; w n and w = v w = nx v i w i : Using this denition, we dene the \norm", or length, of
More informationx 3y 2z = 6 1.2) 2x 4y 3z = 8 3x + 6y + 8z = 5 x + 3y 2z + 5t = 4 1.5) 2x + 8y z + 9t = 9 3x + 5y 12z + 17t = 7
Linear Algebra and its Applications-Lab 1 1) Use Gaussian elimination to solve the following systems x 1 + x 2 2x 3 + 4x 4 = 5 1.1) 2x 1 + 2x 2 3x 3 + x 4 = 3 3x 1 + 3x 2 4x 3 2x 4 = 1 x + y + 2z = 4 1.4)
More informationChapter 6: Orthogonality
Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products
More informationOrthonormal Bases; Gram-Schmidt Process; QR-Decomposition
Orthonormal Bases; Gram-Schmidt Process; QR-Decomposition MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 205 Motivation When working with an inner product space, the most
More informationMATH 1120 (LINEAR ALGEBRA 1), FINAL EXAM FALL 2011 SOLUTIONS TO PRACTICE VERSION
MATH (LINEAR ALGEBRA ) FINAL EXAM FALL SOLUTIONS TO PRACTICE VERSION Problem (a) For each matrix below (i) find a basis for its column space (ii) find a basis for its row space (iii) determine whether
More informationThe following definition is fundamental.
1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic
More informationMath 255 Honors: Gram-Schmidt Orthogonalization on the Space of Polynomials
Math 55 Honors: Gram-Schmidt Orthogonalization on the Space of Polynomials David Moore May, 03 Abstract Gram-Schmidt Orthogonalization is a process to construct orthogonal vectors from some basis for a
More informationMath 3191 Applied Linear Algebra
Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have
More informationMATH 225 Summer 2005 Linear Algebra II Solutions to Assignment 1 Due: Wednesday July 13, 2005
MATH 225 Summer 25 Linear Algebra II Solutions to Assignment 1 Due: Wednesday July 13, 25 Department of Mathematical and Statistical Sciences University of Alberta Question 1. [p 224. #2] The set of all
More informationBASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x
BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,
More informationThe Transpose of a Vector
8 CHAPTER Vectors The Transpose of a Vector We now consider the transpose of a vector in R n, which is a row vector. For a vector u 1 u. u n the transpose is denoted by u T = [ u 1 u u n ] EXAMPLE -5 Find
More informationInner product spaces. Layers of structure:
Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty
More informationy 1 y 2 . = x 1y 1 + x 2 y x + + x n y n y n 2 7 = 1(2) + 3(7) 5(4) = 4.
. Length, Angle, and Orthogonality In this section, we discuss the defintion of length and angle for vectors. We also define what it means for two vectors to be orthogonal. Then we see that linear systems
More informationMA 1B ANALYTIC - HOMEWORK SET 7 SOLUTIONS
MA 1B ANALYTIC - HOMEWORK SET 7 SOLUTIONS 1. (7 pts)[apostol IV.8., 13, 14] (.) Let A be an n n matrix with characteristic polynomial f(λ). Prove (by induction) that the coefficient of λ n 1 in f(λ) is
More informationKernel Method: Data Analysis with Positive Definite Kernels
Kernel Method: Data Analysis with Positive Definite Kernels 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space Kenji Fukumizu The Institute of Statistical Mathematics. Graduate University
More informationHOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)
HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe
More information6 Inner Product and Hilbert Spaces
6 Inner Product and Hilbert Spaces 6. Motivation Of the different p-norms on R n, p = 2 is special. This is because the 2-norm (λ, λ 2,..., λ n ) 2 = λ 2 + λ2 2 + + λ2 n comes from the an inner product
More informationThe Gram Schmidt Process
u 2 u The Gram Schmidt Process Now we will present a procedure, based on orthogonal projection, that converts any linearly independent set of vectors into an orthogonal set. Let us begin with the simple
More informationThe Gram Schmidt Process
The Gram Schmidt Process Now we will present a procedure, based on orthogonal projection, that converts any linearly independent set of vectors into an orthogonal set. Let us begin with the simple case
More informationMath 61CM - Solutions to homework 6
Math 61CM - Solutions to homework 6 Cédric De Groote November 5 th, 2018 Problem 1: (i) Give an example of a metric space X such that not all Cauchy sequences in X are convergent. (ii) Let X be a metric
More informationFourier Series. 1. Review of Linear Algebra
Fourier Series In this section we give a short introduction to Fourier Analysis. If you are interested in Fourier analysis and would like to know more detail, I highly recommend the following book: Fourier
More informationAPPENDIX B GRAM-SCHMIDT PROCEDURE OF ORTHOGONALIZATION. Let V be a finite dimensional inner product space spanned by basis vector functions
301 APPENDIX B GRAM-SCHMIDT PROCEDURE OF ORTHOGONALIZATION Let V be a finite dimensional inner product space spanned by basis vector functions {w 1, w 2,, w n }. According to the Gram-Schmidt Process an
More informationMIDTERM I LINEAR ALGEBRA. Friday February 16, Name PRACTICE EXAM SOLUTIONS
MIDTERM I LIEAR ALGEBRA MATH 215 Friday February 16, 2018. ame PRACTICE EXAM SOLUTIOS Please answer the all of the questions, and show your work. You must explain your answers to get credit. You will be
More informationBeyond Vectors. Hung-yi Lee
Beyond Vectors Hung-yi Lee Introduction Many things can be considered as vectors. E.g. a function can be regarded as a vector We can apply the concept we learned on those vectors. Linear combination Span
More informationInner Product Spaces 5.2 Inner product spaces
Inner Product Spaces 5.2 Inner product spaces November 15 Goals Concept of length, distance, and angle in R 2 or R n is extended to abstract vector spaces V. Sucn a vector space will be called an Inner
More information1. Foundations of Numerics from Advanced Mathematics. Linear Algebra
Foundations of Numerics from Advanced Mathematics Linear Algebra Linear Algebra, October 23, 22 Linear Algebra Mathematical Structures a mathematical structure consists of one or several sets and one or
More informationOctober 25, 2013 INNER PRODUCT SPACES
October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationVector Spaces and SubSpaces
Vector Spaces and SubSpaces Linear Algebra MATH 2076 Linear Algebra Vector Spaces & SubSpaces Chapter 4, Section 1b 1 / 10 What is a Vector Space? A vector space is a bunch of objects that we call vectors
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More information