Linear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products
|
|
- Dominic Stafford
- 5 years ago
- Views:
Transcription
1 Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall A: Inner products In this chapter, the field F = R or C. We regard F equipped with a conjugation χ : F F. If F = R, χ is simply the identity. If F = C, χ(λ) = λ. In both cases, we write χ(λ) = λ.
2 Inner product Let V be a vector space over F. An inner product on V is a nonsingular symmetric bilinear form, : V V F. Symmetry: x, y = y, x for x, y V. Bilinearity: (i) λx + λ x, y = λ x, y + λ x, y, (ii) x, µy + µ y = µ x, y + µ x, y. (ii) follows from (i) and symmetry. Positive definiteness: For every x V. x, x 0 and equality holds if and only if x = 0. 1
3 Standard inner product on R n Consider R n with the standard basis B = {e 1, e 2,..., e n } (where e i has 1 in its i-th position, and 0 elsewhere). The standard inner product on R n is defined by n a i e i, i=1 n b i e i = i=1 n a i b i. i=1 For x = n i=1 x ie i, x 2 = x, x = n x 2 i. i=1 2
4 Unitary space C n Consider C n with standard basis B = {e 1, e 2,..., e n } (where e i has 1 in its i-th position, and 0 elsewhere). The Hermitian inner product on C n is defined by n a i e i, i=1 n b i e i = i=1 n a i b i. i=1 For x = n i=1 x ie i, x 2 = x, x = n x i x i = i=1 n x i 2. i=1 3
5 Examples (1) The space C[a, b] of continuous complex valued functions on a closed interval [a, b] with inner product f, g = b a f(x)g(x)dx. (2) The space l 2. Let F = R or C. An infinite sequence (x n ), n = 0, 1, 2..., in F is square summable if n=0 x n 2 < (convergent). The square summable sequences form the space l 2 with inner product (x n ), (y n ) = x n y n. n=0 This latter series converges by the comparison test since ( x n y n ) 2 0 = x n y n 1 2 ( x n 2 + y n 2 ) = 1 2 ( x n 2 + (y n 2 ). 4
6 Norms The norm of x V is defined by x = x, x. A vector x V is a unit vector if x = 1. A nonzero vector x in a positive definite inner product space can be normalized into a unit vector: x x :=, x, x i.e. x = 1. 5
7 Some inequalities The norm of x V is defined by x = x, x. (1) x 0 and x = 0 if and only if x = 0. (2) λx = λ x for λ F and x V. (3) Cauchy-Schwarz inequality: x, y x y. Equality holds if and only if x and y are linearly dependent. (4) Triangle inequality: x + y x + y. Equality holds if and only if x and y are linearly dependent. (5) Parallelogram law: x + y 2 + x y 2 = 2( x 2 + y 2 ). 6
8 Cauchy-Schwarz inequality Proof. We may assume x and y nonzero. For any λ F, 0 x λy 2 = x λy, x λy = x, x λ x, y λ ( y, x λ y, y ) = x 2 λ x, y λ ( y, x λ y 2). By choosing λ = y, x y 2, we have becomes 0 x 2 y, x y 2 x, y = x 2 y 2 x, y 2 y 2. Therefore, x, y x y. Clearly, equality holds if and only if x λy = 0 for some λ F, i.e., x and y are linearly dependent. 7
9 Polarization identities (1) The real case: x, y = 1 ( u + v 2 u v 2). 4 (2) The complex case: x, y = 1 ( u + v 2 u v 2) + i ( u + iv 2 u iv 2)
10 Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall B: Orthogonality
11 Orthogonality Let V be an inner product space. Two vectors x, y V are orthogonal if x, y = 0. Notation: x y. 1
12 Orthogonal = linearly independent Proposition. Mutually orthogonal nonzero vectors are linearly independent. Proof. Let u 1,..., u s be orthogonal nonzero vectors in an inner product space V. Suppose s i=1 λ iu i = 0. Then for each j = 1,..., s, 0 = s λ i u i, u j = i=1 s λ i u i, u j = λ j u j, u j = λ j = 0 i=1 since u j, u j 0. Therefore, the vectors are linearly independent. 2
13 Orthogonal projection of a vector onto a subspace Let V be a positive definite inner product space, and W V a subspace. For x V, the orthogonal projection onto W is the vector y W for which x y x for every x W. i.e., x, x = x, y. For this, let B = {u 1,..., u s } be a basis of W, and write y = s i=1 a iu i. We require, for each i = 1,..., s, u i, x = u i, s a j u j = j=1 s u i, u j a j. j=1 This amounts to solving the system of linear equations a 1 u 1, x M. =., u s, x a s where M = ( u i, u j ). We shall justify below that M is nonsingular. Granting this, the above equation has a unique solution in a 1,..., a s. This gives the orthogonal projection y of x onto V. 3
14 ( u i, u j ) is a nonsingular matrix Proposition. Let {u 1,..., u n } be a basis of a positive definite inner product space. The matrix M = ( u i, u j ) is nonsingular. Proof. Suppose, for a contradiction, that M is singular. There exists b 1,..., b n F, not all zero, such that b 1 M. = Let y = n j=1 b ju j. It follows from the above that x, y = 0 for every x V. This is clearly impossible since y, y > 0. b n 4
15 Orthogonal complement Let V be an inner product space over F, and V a subspace of V. The orthogonal complement of V is the subspace W := {x V : x, y = 0 for every y W }. Theorem. V = W W. 5
16 Orthogonal decomposition Theorem. V = W W. Proof. (1) W W = 0. (2) For every x V, x = y + (x y), where y is the orthogonal projection of x onto V, and x y W. Therefore, we have an orthogonal direct sum decomposition V = W W. 6
17 Orthogonal decomposition: infinite dimensional case A decomposition V = W W may not hold if V is infinite dimensional. Example. Let V = l 2, and W be the span of all e n = (0,..., 0, 1, 0,...), which contains 1 in its n-th position, and 0 elsewhere. The sequence e 0, e 1,..., e n,..., is orthonormal. Note that W is a proper subspace of l 2, since each element of W, being a linear combination of e 1,..., e n..., has only finitely many nonzero terms. If x = (x n ) W, then for each integer n, 0 = x, e n = x n. This shows that x = 0, and W = {0}. Clearly, l 2 W = W W. 7
18 Orthogonal and orthonormal bases Let V be an inner product space. An orthogonal basis of V is a basis consisting of mutually orthogonal vectors, i.e., a basis B = {u 1,..., u n } of V satisfying u i, u j = 0 for distinct i, j = 1,..., n. An orthonormal basis is an orthogonal basis of unit vectors, i.e., a basis B = {u 1,..., u n } of V satisfying u i, u j = δ i,j for i, j = 1,..., n. 8
19 Existence of orthogonal basis Theorem. A finite dimensional positive definite inner product space contains an orthogonal basis. Proof. (Induction on dimension). This is clearly true for 1-dimensional inner product spaces. Every nonzero vector constitutes an orthogonal basis. Assume this true for all inner product spaces of dimensions < n. Let V be an inner product space of dimension n. Choose a nonzero v V. Then V = Span(v) v, and dim F v = n 1. By inductive hypothesis, v contains an orthogonal basis B. Then {v} B is an orthogonal basis of V. Corollary. A finite dimensional positive definite inner product space contains an orthonormal basis. 9
20 Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall C: Gram-Schmidt orthogonalization In this section, V is a positive definite inner product over R or C.
21 Orthogonal projection onto a vector Lemma. The orthogonal projection of u V along v is the vector u, v v, v v. Proof. Write the orthogonal projection as av for some a F. We require u av, v = 0. From this, u, v a v, v = 0, and a = u, v v, v. 1
22 Orthogonal projection onto a subspace (again) (1) Let W V be a subspace of V, with an orthogonal basis u 1,..., u s. For x V, the orthogonal projection on W is the vector ˆx W such that x ˆx w for every w W. x x ˆx ˆx V Write ˆx = s i=1 λ iu i. For each i = 1, 2..., s, we require x ˆx, u i = 0. This means, x, u i λ i u i, u i = 0, and λ i = x, u i ˆx = s i=1 x, u i u i 2 u i. u i 2. (2) If e 1,..., e s is an orthonormal basis of W V, s ˆx = x, e i e i. i=1 2
23 Bessel s inequality: ˆx x Proof. From x = ˆx + (x ˆx), with ˆx, x ˆx = 0, we have x 2 = ˆx 2 + x ˆx 2 ˆx 2. Therefore, ˆx x. Equality holds if and only if x ˆx = 0, i.e., x W. 3
24 Best approximation ˆx is the best approximation to x in W, in the sense that x ˆx < x w for every w W \ {ˆx}. Proof. For w W, x w = (x ˆx) + (ˆx w). Note that ˆx w W and is orthogonal to x ˆx. Therefore, x w 2 = x ˆx 2 + ˆx w 2 x ˆx 2. Equality holds if and only if ˆx w = 0. Therefore, x ˆx < x w if w W \ {ˆx}. Remark: If V is infinite dimensional, we also call n ˆx = x, e i e i i=1 the Fourier expansion with respect to the orthonormal set e 1,..., e n. The best approximation property and Bessel s inequality are still valid. 4
25 Example: The space l 2 For each integer n 0, let u n be the infinite sequence which has 1 in its n-th position, and 0 elsewhere. The set e 0, e 1,..., e n,... is an orthonormal set. However, the span is not the whole space l 2. Let V be the span of e 1,..., e n. The orthogonal projection of x = (x i ) l 2 is the truncation of x at position n. ˆx = (x 1,..., x n, 0, 0,...), 5
26 Example: V = C[ π, π] The functions 1, cosnx, sinnx, n = 1, 2,..., form an orthogonal set: π π π π π π cosmx cosnxdx = δ m,n π, m, n = 1, 2,..., sinmx sin nxdx = δ m,n π, m, n = 1, 2,..., cosmx sin nxdx = 0, m, n = 0, 1,.... These lead to the Fourier expansions of f(x) C[ π, π]: where a 0 = 1 π a k = 1 π b k = 1 π ˆf = a n (a k coskx + b k sinkx), π π π π π π k=1 f(x)dx, f(x) coskxdx, k = 1, 2,..., f(x) sinkxdx, k = 1, 2,... 6
27 Gram-Schmidt orthogonalization Let v 1,..., v n be given vectors in a positive definite inner product space V. Define a sequence u 1,..., u n as follows. (i) u 1 = v 1, (ii) for k = 2,..., n, u k = v k k 1 i=1 λ k,iu i, where { vk, u i u λ k,i = i if u 2 i 0, 0 if u i = 0. Then u 1,..., u n is a sequence of mutually orthogonal vectors satisfying for each k = 1,..., n. Span(u 1,..., u k ) = Span(v 1,..., v k ) Proposition. If v 1,..., v n is a basis of V, then u 1,..., u n is an orthogonal basis of V. 7
28 QR factorization of a matrix Given an m n matrix over F = R or C, A = ( v 1 v 2... v n ), by reorganizing the vectors obtained in the Gram-Schmidt orthogonalization process, we obtain a factorization A = QR in which (i) Q = ( ũ 1 ũ 2... ũ n ) consists of the vectors u1,..., u n normalized, (ii) R is the upper triangular matrix v u 1 2, u 1 u 1 0 u R = 2 v n, u 1 u 1 v n, u 2 u u n 8
Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product
Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )
More informationMathematics Department Stanford University Math 61CM/DM Inner products
Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector
More informationHILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define
HILBERT SPACES AND THE RADON-NIKODYM THEOREM STEVEN P. LALLEY 1. DEFINITIONS Definition 1. A real inner product space is a real vector space V together with a symmetric, bilinear, positive-definite mapping,
More informationVectors in Function Spaces
Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also
More informationMATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.
MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More informationInner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:
Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationorthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis,
5 Orthogonality Goals: We use scalar products to find the length of a vector, the angle between 2 vectors, projections, orthogonal relations between vectors and subspaces Then we study some applications
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationOrthonormal Bases; Gram-Schmidt Process; QR-Decomposition
Orthonormal Bases; Gram-Schmidt Process; QR-Decomposition MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 205 Motivation When working with an inner product space, the most
More information08a. Operators on Hilbert spaces. 1. Boundedness, continuity, operator norms
(February 24, 2017) 08a. Operators on Hilbert spaces Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ [This document is http://www.math.umn.edu/ garrett/m/real/notes 2016-17/08a-ops
More informationNumerical Linear Algebra
University of Alabama at Birmingham Department of Mathematics Numerical Linear Algebra Lecture Notes for MA 660 (1997 2014) Dr Nikolai Chernov April 2014 Chapter 0 Review of Linear Algebra 0.1 Matrices
More informationLinear Algebra. Session 12
Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)
More informationMath Linear Algebra II. 1. Inner Products and Norms
Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,
More informationOrdinary Differential Equations II
Ordinary Differential Equations II February 23 2017 Separation of variables Wave eq. (PDE) 2 u t (t, x) = 2 u 2 c2 (t, x), x2 c > 0 constant. Describes small vibrations in a homogeneous string. u(t, x)
More informationI teach myself... Hilbert spaces
I teach myself... Hilbert spaces by F.J.Sayas, for MATH 806 November 4, 2015 This document will be growing with the semester. Every in red is for you to justify. Even if we start with the basic definition
More information5 Compact linear operators
5 Compact linear operators One of the most important results of Linear Algebra is that for every selfadjoint linear map A on a finite-dimensional space, there exists a basis consisting of eigenvectors.
More informationMATH 167: APPLIED LINEAR ALGEBRA Chapter 3
MATH 167: APPLIED LINEAR ALGEBRA Chapter 3 Jesús De Loera, UC Davis February 18, 2012 Orthogonal Vectors and Subspaces (3.1). In real life vector spaces come with additional METRIC properties!! We have
More informationINNER PRODUCT SPACE. Definition 1
INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of
More informationWorksheet for Lecture 25 Section 6.4 Gram-Schmidt Process
Worksheet for Lecture Name: Section.4 Gram-Schmidt Process Goal For a subspace W = Span{v,..., v n }, we want to find an orthonormal basis of W. Example Let W = Span{x, x } with x = and x =. Give an orthogonal
More informationThe following definition is fundamental.
1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic
More informationRecall that any inner product space V has an associated norm defined by
Hilbert Spaces Recall that any inner product space V has an associated norm defined by v = v v. Thus an inner product space can be viewed as a special kind of normed vector space. In particular every inner
More informationCHAPTER VIII HILBERT SPACES
CHAPTER VIII HILBERT SPACES DEFINITION Let X and Y be two complex vector spaces. A map T : X Y is called a conjugate-linear transformation if it is a reallinear transformation from X into Y, and if T (λx)
More informationLecture 10: Vector Algebra: Orthogonal Basis
Lecture 0: Vector Algebra: Orthogonal Basis Orthogonal Basis of a subspace Computing an orthogonal basis for a subspace using Gram-Schmidt Orthogonalization Process Orthogonal Set Any set of vectors that
More information4 Hilbert spaces. The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan
The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan Wir müssen wissen, wir werden wissen. David Hilbert We now continue to study a special class of Banach spaces,
More informationMath 407: Linear Optimization
Math 407: Linear Optimization Lecture 16: The Linear Least Squares Problem II Math Dept, University of Washington February 28, 2018 Lecture 16: The Linear Least Squares Problem II (Math Dept, University
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationVector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.
Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar
More informationLinear Algebra. Workbook
Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week: December 4 8 Deadline to hand in the homework: your exercise class on week January 5. Exercises with solutions ) Let H, K be Hilbert spaces, and A : H K be a linear
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More informationInner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that. (1) (2) (3) x x > 0 for x 0.
Inner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that (1) () () (4) x 1 + x y = x 1 y + x y y x = x y x αy = α x y x x > 0 for x 0 Consequently, (5) (6)
More informationLINEAR ALGEBRA W W L CHEN
LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,
More informationChapter 6: Orthogonality
Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products
More informationChapter 6 Inner product spaces
Chapter 6 Inner product spaces 6.1 Inner products and norms Definition 1 Let V be a vector space over F. An inner product on V is a function, : V V F such that the following conditions hold. x+z,y = x,y
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationMath 489AB A Very Brief Intro to Fourier Series Fall 2008
Math 489AB A Very Brief Intro to Fourier Series Fall 8 Contents Fourier Series. The coefficients........................................ Convergence......................................... 4.3 Convergence
More information5. Orthogonal matrices
L Vandenberghe EE133A (Spring 2017) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal
More informationLinear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall 2011
Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 1A: Vector spaces Fields
More informationMath 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.
Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,
More informationFourier and Wavelet Signal Processing
Ecole Polytechnique Federale de Lausanne (EPFL) Audio-Visual Communications Laboratory (LCAV) Fourier and Wavelet Signal Processing Martin Vetterli Amina Chebira, Ali Hormati Spring 2011 2/25/2011 1 Outline
More informationLecture Notes for Inf-Mat 3350/4350, Tom Lyche
Lecture Notes for Inf-Mat 3350/4350, 2007 Tom Lyche August 5, 2007 2 Contents Preface vii I A Review of Linear Algebra 1 1 Introduction 3 1.1 Notation............................... 3 2 Vectors 5 2.1 Vector
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationMTH 2032 SemesterII
MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents
More informationOctober 25, 2013 INNER PRODUCT SPACES
October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal
More informationMath 413/513 Chapter 6 (from Friedberg, Insel, & Spence)
Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) David Glickenstein December 7, 2015 1 Inner product spaces In this chapter, we will only consider the elds R and C. De nition 1 Let V be a vector
More informationDot product and linear least squares problems
Dot product and linear least squares problems Dot Product For vectors u,v R n we define the dot product Note that we can also write this as u v = u,,u n u v = u v + + u n v n v v n = u v + + u n v n The
More informationTypical Problem: Compute.
Math 2040 Chapter 6 Orhtogonality and Least Squares 6.1 and some of 6.7: Inner Product, Length and Orthogonality. Definition: If x, y R n, then x y = x 1 y 1 +... + x n y n is the dot product of x and
More informationMATH 532: Linear Algebra
MATH 532: Linear Algebra Chapter 5: Norms, Inner Products and Orthogonality Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2015 fasshauer@iit.edu MATH 532 1 Outline
More informationMATH 235: Inner Product Spaces, Assignment 7
MATH 235: Inner Product Spaces, Assignment 7 Hand in questions 3,4,5,6,9, by 9:3 am on Wednesday March 26, 28. Contents Orthogonal Basis for Inner Product Space 2 2 Inner-Product Function Space 2 3 Weighted
More informationMAT Linear Algebra Collection of sample exams
MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationFall TMA4145 Linear Methods. Exercise set Given the matrix 1 2
Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 07 Exercise set Please justify your answers! The most important part is how you arrive at an
More informationA Primer in Econometric Theory
A Primer in Econometric Theory Lecture 1: Vector Spaces John Stachurski Lectures by Akshay Shanker May 5, 2017 1/104 Overview Linear algebra is an important foundation for mathematics and, in particular,
More informationBASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x
BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,
More informationNORMS ON SPACE OF MATRICES
NORMS ON SPACE OF MATRICES. Operator Norms on Space of linear maps Let A be an n n real matrix and x 0 be a vector in R n. We would like to use the Picard iteration method to solve for the following system
More informationChapter 4 Euclid Space
Chapter 4 Euclid Space Inner Product Spaces Definition.. Let V be a real vector space over IR. A real inner product on V is a real valued function on V V, denoted by (, ), which satisfies () (x, y) = (y,
More informationMA20216: Algebra 2A. Notes by Fran Burstall
MA20216: Algebra 2A Notes by Fran Burstall Corrections by: Callum Kemp Carlos Galeano Rios Kate Powell Tobias Beith Krunoslav Lehman Pavasovic Dan Corbie Phaidra Anastasiadou Louise Hannon Vlad Brebeanu
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More information6 Inner Product Spaces
Lectures 16,17,18 6 Inner Product Spaces 6.1 Basic Definition Parallelogram law, the ability to measure angle between two vectors and in particular, the concept of perpendicularity make the euclidean space
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationLinear Algebra 1. M.T.Nair Department of Mathematics, IIT Madras. and in that case x is called an eigenvector of T corresponding to the eigenvalue λ.
Linear Algebra 1 M.T.Nair Department of Mathematics, IIT Madras 1 Eigenvalues and Eigenvectors 1.1 Definition and Examples Definition 1.1. Let V be a vector space (over a field F) and T : V V be a linear
More informationMath 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008
Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures
More informationLecture notes on Quantum Computing. Chapter 1 Mathematical Background
Lecture notes on Quantum Computing Chapter 1 Mathematical Background Vector states of a quantum system with n physical states are represented by unique vectors in C n, the set of n 1 column vectors 1 For
More informationSection 7.5 Inner Product Spaces
Section 7.5 Inner Product Spaces With the dot product defined in Chapter 6, we were able to study the following properties of vectors in R n. ) Length or norm of a vector u. ( u = p u u ) 2) Distance of
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More informationPART II : Least-Squares Approximation
PART II : Least-Squares Approximation Basic theory Let U be an inner product space. Let V be a subspace of U. For any g U, we look for a least-squares approximation of g in the subspace V min f V f g 2,
More informationLecture 1: Review of linear algebra
Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, least-squares and least-norm solutions Subspaces, basis, and dimension Change of basis and similarity transformations
More informationHilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space.
Hilbert Spaces Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space. Vector Space. Vector space, ν, over the field of complex numbers,
More informationMATH Linear Algebra
MATH 304 - Linear Algebra In the previous note we learned an important algorithm to produce orthogonal sequences of vectors called the Gramm-Schmidt orthogonalization process. Gramm-Schmidt orthogonalization
More informationMathematics of Information Spring semester 2018
Communication Technology Laboratory Prof. Dr. H. Bölcskei Sternwartstrasse 7 CH-809 Zürich Mathematics of Information Spring semester 08 Solution to Homework Problem Overcomplete expansion in R a) Consider
More informationLecture 4 Orthonormal vectors and QR factorization
Orthonormal vectors and QR factorization 4 1 Lecture 4 Orthonormal vectors and QR factorization EE263 Autumn 2004 orthonormal vectors Gram-Schmidt procedure, QR factorization orthogonal decomposition induced
More informationLINEAR ALGEBRA SUMMARY SHEET.
LINEAR ALGEBRA SUMMARY SHEET RADON ROSBOROUGH https://intuitiveexplanationscom/linear-algebra-summary-sheet/ This document is a concise collection of many of the important theorems of linear algebra, organized
More informationSpectral Theory, with an Introduction to Operator Means. William L. Green
Spectral Theory, with an Introduction to Operator Means William L. Green January 30, 2008 Contents Introduction............................... 1 Hilbert Space.............................. 4 Linear Maps
More informationFunctional Analysis HW #5
Functional Analysis HW #5 Sangchul Lee October 29, 2015 Contents 1 Solutions........................................ 1 1 Solutions Exercise 3.4. Show that C([0, 1]) is not a Hilbert space, that is, there
More informationLinear Algebra. Paul Yiu. 6D: 2-planes in R 4. Department of Mathematics Florida Atlantic University. Fall 2011
Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 6D: 2-planes in R 4 The angle between a vector and a plane The angle between a vector v R n and a subspace V is the
More informationFunctional Analysis. James Emery. Edit: 8/7/15
Functional Analysis James Emery Edit: 8/7/15 Contents 0.1 Green s functions in Ordinary Differential Equations...... 2 0.2 Integral Equations........................ 2 0.2.1 Fredholm Equations...................
More informationDiagonalizing Matrices
Diagonalizing Matrices Massoud Malek A A Let A = A k be an n n non-singular matrix and let B = A = [B, B,, B k,, B n ] Then A n A B = A A 0 0 A k [B, B,, B k,, B n ] = 0 0 = I n 0 A n Notice that A i B
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationChapter 8 Integral Operators
Chapter 8 Integral Operators In our development of metrics, norms, inner products, and operator theory in Chapters 1 7 we only tangentially considered topics that involved the use of Lebesgue measure,
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationSYLLABUS. 1 Linear maps and matrices
Dr. K. Bellová Mathematics 2 (10-PHY-BIPMA2) SYLLABUS 1 Linear maps and matrices Operations with linear maps. Prop 1.1.1: 1) sum, scalar multiple, composition of linear maps are linear maps; 2) L(U, V
More informationPart 1a: Inner product, Orthogonality, Vector/Matrix norm
Part 1a: Inner product, Orthogonality, Vector/Matrix norm September 19, 2018 Numerical Linear Algebra Part 1a September 19, 2018 1 / 16 1. Inner product on a linear space V over the number field F A map,
More informationRecall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning:
Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u 2 1 + u 2 2 = u 2. Geometric Meaning: u v = u v cos θ. u θ v 1 Reason: The opposite side is given by u v. u v 2 =
More informationThroughout these notes we assume V, W are finite dimensional inner product spaces over C.
Math 342 - Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal
More informationMIDTERM I LINEAR ALGEBRA. Friday February 16, Name PRACTICE EXAM SOLUTIONS
MIDTERM I LIEAR ALGEBRA MATH 215 Friday February 16, 2018. ame PRACTICE EXAM SOLUTIOS Please answer the all of the questions, and show your work. You must explain your answers to get credit. You will be
More informationChapter 5. Basics of Euclidean Geometry
Chapter 5 Basics of Euclidean Geometry 5.1 Inner Products, Euclidean Spaces In Affine geometry, it is possible to deal with ratios of vectors and barycenters of points, but there is no way to express the
More informationSection 6.4. The Gram Schmidt Process
Section 6.4 The Gram Schmidt Process Motivation The procedures in 6 start with an orthogonal basis {u, u,..., u m}. Find the B-coordinates of a vector x using dot products: x = m i= x u i u i u i u i Find
More informationx 3y 2z = 6 1.2) 2x 4y 3z = 8 3x + 6y + 8z = 5 x + 3y 2z + 5t = 4 1.5) 2x + 8y z + 9t = 9 3x + 5y 12z + 17t = 7
Linear Algebra and its Applications-Lab 1 1) Use Gaussian elimination to solve the following systems x 1 + x 2 2x 3 + 4x 4 = 5 1.1) 2x 1 + 2x 2 3x 3 + x 4 = 3 3x 1 + 3x 2 4x 3 2x 4 = 1 x + y + 2z = 4 1.4)
More informationI. Multiple Choice Questions (Answer any eight)
Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam - Course Instructor : Prashanth L.A. Date : Sep-24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY
More informationSolutions to Review Problems for Chapter 6 ( ), 7.1
Solutions to Review Problems for Chapter (-, 7 The Final Exam is on Thursday, June,, : AM : AM at NESBITT Final Exam Breakdown Sections % -,7-9,- - % -9,-,7,-,-7 - % -, 7 - % Let u u and v Let x x x x,
More informationReal Analysis Notes. Thomas Goller
Real Analysis Notes Thomas Goller September 4, 2011 Contents 1 Abstract Measure Spaces 2 1.1 Basic Definitions........................... 2 1.2 Measurable Functions........................ 2 1.3 Integration..............................
More informationCHAPTER II HILBERT SPACES
CHAPTER II HILBERT SPACES 2.1 Geometry of Hilbert Spaces Definition 2.1.1. Let X be a complex linear space. An inner product on X is a function, : X X C which satisfies the following axioms : 1. y, x =
More informationMathematical foundations - linear algebra
Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More informationInner Product Spaces
Inner Product Spaces Introduction Recall in the lecture on vector spaces that geometric vectors (i.e. vectors in two and three-dimensional Cartesian space have the properties of addition, subtraction,
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More information(v, w) = arccos( < v, w >
MA322 F all203 Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: For all v,
More informationLinear Normed Spaces (cont.) Inner Product Spaces
Linear Normed Spaces (cont.) Inner Product Spaces October 6, 017 Linear Normed Spaces (cont.) Theorem A normed space is a metric space with metric ρ(x,y) = x y Note: if x n x then x n x, and if {x n} is
More information