# Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence)

Save this PDF as:

Size: px
Start display at page:

Download "Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence)"

## Transcription

1 Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) David Glickenstein December 7, Inner product spaces In this chapter, we will only consider the elds R and C. De nition 1 Let V be a vector space over F = R or C. An inner product on V is a function V V! F; denoted (x; y)! hx; yi ; such that the following hold for all x; y; z 2 V and c 2 F : 1. hx + z; yi = hx; yi + hz; yi. 2. hcx; yi = c hx; yi : 3. hy; xi = hx; yi where the bar denotes complex conjugation. 4. hx; xi > 0 if x 6= 0: Here are some observations: The rst two conditions could be called being linear in the rst component. The third condition is called being conjugate symmetric (or symmetric if F = R). Linearity in the rst component and conjugate symmetry imply linearity in the second component, and being linear in both components is called being bilinear. Notice that conjugate symmetry implies that hx; xi 2 R even if F = C since hx; xi = hx; xi: D The second condition also implies that ~ 0;~0E = 0; which together with the fourth condition is called being positive de nite. From all of this, we could have just speci ed that h; i is bilinear, conjugate symmetric, and positive de nite. 1

2 We often write kvk 2 to represent hv; vi : Since kvk 2 0; it has a unique positive square root that we call kvk : Example 2 The rst example is the dot product for R n ; where if x = (x 1 ; : : : ; x n ) and y = (y 1 ; : : : ; y n ) then hx; yi = x y = nx x i y i : Example 3 The standard inner product on C n is Check the properties. hx; yi = nx x i y i : Example 4 Given any inner product h; i ; we can multiply it by a positive real number r > 0 to get another inner product hx; yi 0 = r hx; yi : Note that it would not be an inner product if r 0 or r is not a real number. Example 5 For continuous, real-valued functions on [0; 1] ; there is an inner product hf; gi = Z 1 0 f (t) g (t) dt: Note that it is important that the functions be continuous to ensure that hf; fi = R 1 0 f (t)2 dt > 0 if f 6= ~0: Example 6 Consider the vector space C nn : We de ne the conjugate transpose, or adjoint, A of a matrix A by specifying the entries as: We can now de ne an inner product by (A ) ij = A ji : ha; Bi = tr (B A) : Let s check it is an inner product. First, ha + A 0 ; Bi = tr (B (A + A 0 )) = tr (B A + B A 0 ) = tr (B A) + tr (B A 0 ) = ha; Bi + ha 0 ; Bi : Also, hb; Ai = tr (A B) ha; Bi = tr B A = tr (A B) : Also, ha; Ai = tr (A A) = X i X A ija ji = X j j;i A ji A ji = X j;i ja ji j 2 and so this is nonnegative and equals zero if and only if A = 0: 2

3 De nition 7 A vector space V together with an inner product is called an inner product space. If the eld is C then we call it a complex inner product space, and if the eld is R we call it a real inner product space. The following are basic properties of inner product spaces. Theorem 8 Let V be an inner product space. Then for x; y; z 2 V and c 2 F the following statements are true: 1. hx; y + zi = hx; yi + hx; zi : 2. hx; cyi = c hx; yi : D E D E 3. x;~0 = ~ 0; x = 0: 4. hx; xi = 0 if and only if x = ~0: 5. If hx; yi = hx; zi for all x 2 V; then y = z: Proof. Most of these follow pretty easily. The last one can be shown as follows. Suppose hx; yi = hx; zi for all x 2 V: Then hx; y zi = 0 for all x: In particular, taking x = y z; we get that hy z; y zi = 0: But that implies y z = ~0: Note that the rst two statements in the above theorem are called being conjugate linear in the second component. Recall the de nition of the length, or norm: kxk = p hx; xi: This generalizes the Euclidean norm k(x 1 ; : : : ; x n )k = p x x2 n: Many properties are still true for inner product spaces: Theorem 9 Let V be an inner product space over F = R or C. The for x; y 2 V and c 2 F; the following are true: 1. kcxk = jcj kxk : 2. kxk = 0 i x = ~0: In general, kxk 0: 3. (Cauchy-Schwarz inequality) jhx; yij kxk kyk : 4. (Triangle inequality) kx + yk kxk + kyk : Proof. We will just prove the last two. Consider x 0 kx cyk 2 Notice that if we take c = hx;yi kyk 2 = hx cy; x cyi = kxk 2 hcy; xi hx; cyi + kcyk 2 = kxk 2 2 Re c hx; yi + jcj 2 kyk 2 : then cy and notice that 0 kxk 2 2 jhx; yij2 jhx; yij 2 kyk 2 + kyk 2 = kxk 2 2 jhx; yij kyk 2 ; 3

4 which implies the Cauchy-Schwarz inequality. We can now show kx + yk 2 = hx + y; x + yi = kxk Re hx; yi + kyk 2 kxk kxk kyk + kyk 2 = (kxk + kyk) 2 since 2a a 2 + b 2 = ja + bij 2 and so 2 Re hx; yi jhx; yij kxk kyk : De nition 10 Let V be an inner product space. Vectors x and y in V are orthogonal ( perpendicular) if hx; yi = 0: As subset S V is orthogonal if any two distinct vectors in S are orthogonal. A vector x in V is a unit vector if kxk = 1: A subset S V is orthonormal if S is orthogonal and consists entirely of unit vectors. Note that S = fx 1 ; : : : ; x k g is orthonormal i hx i ; x j i = ij : Also note that we can make an orthonormal set from an orthogonal set by replacing D each E vector x by 1 kxk x: This will not change the orthogonality since x kxk ; y kyk = hx; yi since kyk 2 R. We call this process normalizing the set. 1 kxkkyk Proposition 11 If V is an inner product space and S V is orthogonal and nonzero, then S is linearly independent. Proof. We rst note that if S is not the set consisting only of zero, then zero cannot be in S: Suppose a 1 x a k x k = ~0 for scalars a 1 ; : : : ; a k and vectors x 1 ; : : : ; x k in S: Then we see that 0 = ha 1 x a k x k ; x i i = a i kx i k 2 and since kx i k 2 6= 0; we must have a i = 0. This can be done for all i: 2 Problems FIS Section 6.1 exercises 2, 3, 8, 10, 11, 16, 17, 19, 20, 22 3 Orthonormal bases De nition 12 Let V be an inner product space. A subset of V is an orthonormal basis for V if it is an ordered basis that is orthonormal. The standard basis is orthonormal for the usual inner product. So is any rotation of the standard basis! Orthonormal bases make it easier to write the coe cients than in any old basis. 4

5 Theorem 13 Let V be an inner product space and S = fv 1 ; v 2 ; : : : ; v k g be an orthogonal subset of V consisting of nonzero vectors. If y 2 span S; then y = hy; v i i hv i ; v i i v i: Proof. Since y 2 span S; we must have that there exist scalars a 1 ; : : : ; a k such that y = a i v i : We can now take the inner product with v j for j = 1; : : : ; k and nd that * + hy; v j i = a i v i ; v j and so (since kv j k 6= 0), a j = hy;vji hv j;v ji : y = = a i hv i ; v j i = a j hv j ; v j i hy; v i i hv i ; v i i v i: Note that if S is orthonormal, then the denominators are all 1: We can always take a linearly independent set and use it to nd an orthogonal set with the same span. We do this by considering orthogonal projections of vectors onto a subspace. Theorem 14 Let W be a nite dimensional subspace of the inner product space V: Then for a vector y 2 V; there is a unique vector u 2 W that minimizes ky wk 2 for all w 2 W: Proof. Suppose there is a u 2 W such that hw; y if w 2 W (and hence so is u w), ui = 0 for any w 2 W: Then ky wk 2 = ku + (y u) wk 2 = hu w + (y u) ; u w + (y u)i = ku wk 2 + hu w; y ui + hy u; u wi + ky uk 2 = ku wk 2 + ky uk 2 ky uk 2 : We can do this if W is nite dimensional using the following theorem. 5

6 De nition 15 We call the assignment of u the orthogonal projection of y onto W; denoted u = P W (y) : It is an important fact that hy P W (y) ; wi = 0 for all w 2 W: De nition 16 The orthogonal complement of W; written W? (pronounced W perp ), is the set of all vectors v 2 V such that hv; wi for all w 2 W. Note that W? is a vector space. Proposition 17 W? is a vector space. D E Proof. It is straightforward to see that ~ 0; w = 0 for all w 2 W; so ~0 2 W? : if v; u 2 W? and c 2 F then hcv + u; wi = c hv; wi = hu; wi = 0 so cv + u 2 W? : We can construct an orthogonal set from a linearly independent set by starting with the rst vector and then projecting the next vector into Theorem 18 Let V be an inner product space and S = fw 1 ; : : : ; w n g be a linearly independent subset of V: De ne S 0 = fv 1 ; : : : ; v n g by v 1 = w 1 and 1 v k = w k j=1 hw k ; v j i hv j ; v j i v j for k = 2; : : : ; n: Then S 0 span S 0 = span S: is an orthogonal set of nonzero vectors such that Proof. We show inductively that v k+1 is orthogonal to v 1 ; : : : ; v k : It is clear that hw 2 ; v 1 i hv 2 ; v 1 i = w 2 hv 1 ; v 1 i v 1; v 1 = hw 2 ; v 1 i hw 2 ; v 1 i hv 1 ; v 1 i hv 1; v 1 i = 0: We then can use the inductive hypothesis to assume hv i ; v j i = 0 for i; j k and see that * + 1 hw k ; v j i hv k ; v i i = w k hv j ; v j i v j; v i = hw k ; v i i j=1 hw k ; v i i hv i ; v i i hv i; v i i = 0: Thus S 0 is orthogonal. Hence S 0 is linearly independent and since each element of S 0 is in the span of S; span S 0 span S; and hence span S 0 = span S (since they have the same dimension). Note: this process of producing an orthogonal set is called the Gram-Schmidt process. 6

7 Theorem 19 Suppose that S = fv 1 ; : : : ; v k g is an orthonormal set in a n- dimensional inner product space V: Then 1. S can be extended to an orthonormal basis fv 1 ; : : : ; v k ; v k+1; : : : ; v n g for V: 2. If W = span S; then S 1 = fv k+1 ; : : : ; v n g is an orthonormal basis for W? : 3. If W is any subspace of V; then dim V = dim W + dim W? : Proof. By the replacement theorem, S can be extended into a basis, and then the Gram-Schmidt process can be used to turn this into an orthogonal set. Then normalizing gives an orthonormal set. S 1 is clearly a linearly independent subset of W? : Since fv 1 ; : : : ; v n g is a basis, any vector in W? can be written as a linear combination of these vectors. However, since w 2 W? satis es hw; v i i = 0 for i = 1; : : : ; k; w is in the span of S 1 ; hence S 1 is a basis. The dimension statement is clear now that we know that S is a basis for S; S 0 is a basis for W? ; and fv 1 ; : : : ; v n g is a basis for V: 4 Problems FIS Section 6.2 exercises 4, 5, 7, 8, 10, 13, Adjoints and eigenvalues De nition 20 Suppose T is a linear operator on an inner product space V: If T is a linear operator on V such that ht (x) ; yi = hx; T (y)i for all x; y 2 V; we say T is the adjoint of T: We read T as T star. Theorem 21 Let V be a nite-dimensional inner product space and let T be a linear operator on V: Then there exists a unique function T : V! V that for all x; y 2 V; ht (x) ; yi = hx; T (y)i : Furthermore, T is linear and hence the adjoint of T: In order to prove this theorem, we need to show that every linear map on a nite-dimensional vector space can be represented in terms of the inner product. We will then use this idea to construct the adjoint. Theorem 22 Let V be a nite dimensional vector space over F and let g : V! F be a linear transformation. Then there exists a unique y 2 V such that g (x) = hx; yi for all x 2 V: 7

8 Proof. Let = fv 1 ; : : : ; v n g be an orthonormal basis for V: For any x 2 V; we must have y = P b i v i for some b i ; so in order for g (x) = hx; yi we must have that g (v i ) = hv i ; yi = b i : Hence any y that satis es the equation must have b i = g (v i ) (this proves uniqueness). We now con rm that y = P g (v i )v i satis es the theorem: if x = P a i v i then * n + X nx hx; yi = a i v i ; g (v j )v j = = nx j=1 j=1 nx a i g (v j ) hv i ; v j i nx a i g (v i ) = g (x) : Proof of Theorem 21. Let = fv 1 ; : : : ; v n g ; be an orthonormal basis for V and de ne T i : V! F by T i (x) = ht (x) ; v i i : By Theorem 22, there exists a unique y i 2 V such that T i (x) = hx; y i i ; and so ht (x) ; v i i = hx; y i i : We can thus de ne a linear tranformation T by de ning it on the basis to be T (v i ) = y i and extending it linearly to a transformation on V: Now, for any vector y 2 V; y = P b i v i for some scalars b 1 ; : : : ; b n and we check that ht (x) ; yi = DT (x) ; X E b i v i = X bi ht (x) ; v i i = X bi hx; T (v i )i = X hx; b i T (v i )i = hx; T (y)i : T is unique since if there were another linear transformation U satisfying the properties, then for any y 2 V; hx; T (y)i = ht (x) ; yi = hx; U (y)i must be true for all x 2 V; implying that T (y) = U (y) : (See 6.1, problem 9.) Notice that the adjoint works the other way as well: hx; T (y)i = ht (y) ; xi = hy; T (x)i = ht (x) ; yi : 8

9 Note our proof of the existence of an adjoint used a basis, and it turns out that for a linear operator on an in nite dimensional vector space, the existence of an adjoint is not guaranteed. However, most properties we will derive are true if the adjoint exists (sometimes it does). Recall that for a matrix A; we denote its conjugate transpose by A (conjugate transpose means we take a transpose and replace each entry with its complex conjugate). This is related to the adjoint. Theorem 23 Let V be a nite dimensional inner product space and be an orthonormal basis for V: If T is a linear operator on V; then [T ] = [T ] : It might help to be clear about the content here. If one writes the matrix for the adjoint transformation T, this matrix is the same as the conjugate transpose of the matrix for the transformation T (provided the basis used in both cases is orthonormal). Proof. If we write A = [T ] and B = [T ] then by orthonormality we have that B ij = ht (v j ) ; v i i = hv j ; T (v i )i = ht (v i ) ; v j i = A ji : It follows that B = A : Corollary 24 Left multiplication by an n n matrix A satis es L A = L A: Theorem 25 Let V be an inner product space and T; U be linear operators on V that have an adjoint (this is always true if V is nite dimensional). Then 1. (T + U) = T + U 2. (ct ) = ct for any c 2 F 3. (T U) = U T 4. T = T 5. I V = I V : Proof. The proofs are pretty straightforward. We prove the rst and third: for any x; y 2 V h(t + U) x; yi = ht (x) + U (x) ; yi = ht (x) ; yi + hu (x) ; yi = hx; T (y)i + hx; U (y)i = hx; T (y) + U (y)i = hx; (T + U ) (y)i : 9

10 Also, ht U (x) ; yi = ht (U (x)) ; yi = hu (x) ; T (y)i = hx; U T (y)i : 6 Problems FIS Section 6.3 exercises 4, 6, 8, 10, 12, 13, 15 7 Schur s theorem and the (baby) spectral theorem We would like to understand when there exists an orthonormal basis of eigenvectors. We rst have Schur s theorem, that says that we can represent a linear transformation by an upper triangular matrx. Theorem 26 (Schur) Let T be a linear operator on a nite dimensional inner product space V: Suppose the characteristic polynomial of T splits. The there exists an orthonormal basis of V such that [T ] is upper triangular. The main idea behind the proof is that we can restrict T to a subspace. If W is a subspace of V and T (W ) W; then we say that W is T -invariant. Hence there is a linear operator T W 2 L (W ) that is just T W (x) = T (x) : If W is T -invariant and is a basis for V that is an extension of a basis for W; then [T ] has block form [TW ] 0 where the s are unknown but generally not zero. We now see that if z is an eigenvector of T and if z? = fx 2 V : hx; zi = 0g ; then z? is T -invariant, i.e., ht (y) ; zi = hy; T (z)i = hy; zi = hy; zi = 0: Proof. We rst notice that the characteristic polynomial of T satis es det (T I) = det T I since determinant is invariant under trace. Hence if the characteristic polynomial of T splits, then there is an eigenvector z of T : Then z? is T -invariant, and so we can use the inductive hypothesis to show that [T ] is upper triangular. This completes the proof. Notice the following consequence: 10

11 Theorem 27 Suppose T 2 L (V ) for a nite dimensional inner product space V: If T is self-adjoint, i.e., T = T, and its characteristic polynomial splits, then V has an orthonormal basis of eigenvectors of T: Proof. Since T = T ; we have that [T ] = [T ] : If we choose the basis from the Schur theorem, [T ] is upper triangular and [T ] is lower triangular and since [T ] = [T ] = [T ] we must have that [T ] is triangular. Note that we needed that the characteristic polynomials splits, which is not known for real inner product spaces, so we need the following lemma to complete the proof. Lemma 28 Let T be a self-adjoint operator on a nite dimensional inner product space V. Then 1. T has real eigenvalues 2. The characteristic polynomial of T splits. Proof. If is an eigenvalue with eigenvector v; then hv; vi = ht (v) ; vi = hv; T (v)i = hv; T (v)i = hv; vi = hv; vi and so = and is real. The second statement follows from the fundamental theorem of algebra if F = C. If F = R then we can consider A = [T ] for some basis ; and then consider the linear transformation L A : C n! C n given by L A (x) = Ax: Since T is self-adjoint, A = A and hence L A is self-adjoint and hence has real eigenvalues. Since all polynomials split over the complex numbers, the characteristic polynomial for L A splits, and since the eigenvalues are real, the polynomial has real coe cients. The characteristic polynomial for L A is the same as that for T; and since the eigenvalues are real, that means that the characteristic polynomial for T splits. Remark 29 The theorem above is if and only if, actually, for real inner product spaces. For complex inner product spaces, self-adjoint can be relaxed to normal, which means that T and T commute, i.e. T T = T T: For a normal operator, any eigenvector x of T; with eigenvalue ; is an eigenvector for T since 0 = ht x x; T x xi = hx; T T xi hx; T xi ht x; xi + jj 2 hx; xi = hx; T T xi ht x; xi hx; T xi + jj 2 hx; xi = T x x; T x x and so x is an eigenvector for T with eigenvalue : 8 Problems 11

### 4.3 - Linear Combinations and Independence of Vectors

- Linear Combinations and Independence of Vectors De nitions, Theorems, and Examples De nition 1 A vector v in a vector space V is called a linear combination of the vectors u 1, u,,u k in V if v can be

### Math Linear Algebra II. 1. Inner Products and Norms

Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,

### Introduction to Linear Algebra. Tyrone L. Vincent

Introduction to Linear Algebra Tyrone L. Vincent Engineering Division, Colorado School of Mines, Golden, CO E-mail address: tvincent@mines.edu URL: http://egweb.mines.edu/~tvincent Contents Chapter. Revew

### Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

### Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

### Functional Analysis: Assignment Set # 3 Spring 2009 Professor: Fengbo Hang February 25, 2009

duardo Corona Functional Analysis: Assignment Set # 3 Spring 9 Professor: Fengbo Hang February 5, 9 C6. Show that a norm that satis es the parallelogram identity: comes from a scalar product. kx + yk +

### Homework 11 Solutions. Math 110, Fall 2013.

Homework 11 Solutions Math 110, Fall 2013 1 a) Suppose that T were self-adjoint Then, the Spectral Theorem tells us that there would exist an orthonormal basis of P 2 (R), (p 1, p 2, p 3 ), consisting

### MTH 5102 Linear Algebra Practice Final Exam April 26, 2016

Name (Last name, First name): MTH 5 Linear Algebra Practice Final Exam April 6, 6 Exam Instructions: You have hours to complete the exam. There are a total of 9 problems. You must show your work and write

### 4 Linear Algebra Review

Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the text 1 Vectors and Matrices For the context of data analysis, the

### Chapter 6 Inner product spaces

Chapter 6 Inner product spaces 6.1 Inner products and norms Definition 1 Let V be a vector space over F. An inner product on V is a function, : V V F such that the following conditions hold. x+z,y = x,y

### Mathematics Department Stanford University Math 61CM/DM Inner products

Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector

### 6 Inner Product Spaces

Lectures 16,17,18 6 Inner Product Spaces 6.1 Basic Definition Parallelogram law, the ability to measure angle between two vectors and in particular, the concept of perpendicularity make the euclidean space

### Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v

### October 25, 2013 INNER PRODUCT SPACES

October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal

### DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

### MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

### Vector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.

Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar

### Math 702 Problem Set 10 Solutions

Math 70 Problem Set 0 Solutions Unless otherwise stated, X is a complex Hilbert space with the inner product h; i the induced norm k k. The null-space range of a linear map T W X! X are denoted by N.T

### Functional Analysis Review

Functional Analysis Review Lorenzo Rosasco slides courtesy of Andre Wibisono 9.520: Statistical Learning Theory and Applications September 9, 2013 1 2 3 4 Vector Space A vector space is a set V with binary

### Lecture notes: Applied linear algebra Part 1. Version 2

Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and

### Spectral Theorem for Self-adjoint Linear Operators

Notes for the undergraduate lecture by David Adams. (These are the notes I would write if I was teaching a course on this topic. I have included more material than I will cover in the 45 minute lecture;

### LEARNING OBJECTIVES FOR THIS CHAPTER

CHAPTER 6 Woman teaching geometry, from a fourteenth-century edition of Euclid s geometry book. Inner Product Spaces In making the definition of a vector space, we generalized the linear structure (addition

### 1. General Vector Spaces

1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

### Linear Algebra Massoud Malek

CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

### Section 7.5 Inner Product Spaces

Section 7.5 Inner Product Spaces With the dot product defined in Chapter 6, we were able to study the following properties of vectors in R n. ) Length or norm of a vector u. ( u = p u u ) 2) Distance of

### Algebra II. Paulius Drungilas and Jonas Jankauskas

Algebra II Paulius Drungilas and Jonas Jankauskas Contents 1. Quadratic forms 3 What is quadratic form? 3 Change of variables. 3 Equivalence of quadratic forms. 4 Canonical form. 4 Normal form. 7 Positive

### Math 108b: Notes on the Spectral Theorem

Math 108b: Notes on the Spectral Theorem From section 6.3, we know that every linear operator T on a finite dimensional inner product space V has an adjoint. (T is defined as the unique linear operator

### Typical Problem: Compute.

Math 2040 Chapter 6 Orhtogonality and Least Squares 6.1 and some of 6.7: Inner Product, Length and Orthogonality. Definition: If x, y R n, then x y = x 1 y 1 +... + x n y n is the dot product of x and

### MATRICES ARE SIMILAR TO TRIANGULAR MATRICES

MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,

### INNER PRODUCT SPACE. Definition 1

INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of

### Linear Algebra 2 Spectral Notes

Linear Algebra 2 Spectral Notes In what follows, V is an inner product vector space over F, where F = R or C. We will use results seen so far; in particular that every linear operator T L(V ) has a complex

### There are two things that are particularly nice about the first basis

Orthogonality and the Gram-Schmidt Process In Chapter 4, we spent a great deal of time studying the problem of finding a basis for a vector space We know that a basis for a vector space can potentially

### Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

### Chapter 4 Euclid Space

Chapter 4 Euclid Space Inner Product Spaces Definition.. Let V be a real vector space over IR. A real inner product on V is a real valued function on V V, denoted by (, ), which satisfies () (x, y) = (y,

### Lecture Notes 1: Vector spaces

Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector

### Exercise Sheet 1.

Exercise Sheet 1 You can download my lecture and exercise sheets at the address http://sami.hust.edu.vn/giang-vien/?name=huynt 1) Let A, B be sets. What does the statement "A is not a subset of B " mean?

### Math 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.

Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses

### Chapter 3 Transformations

Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases

### LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM

LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM Unless otherwise stated, all vector spaces in this worksheet are finite dimensional and the scalar field F is R or C. Definition 1. A linear operator

### Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning:

Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u 2 1 + u 2 2 = u 2. Geometric Meaning: u v = u v cos θ. u θ v 1 Reason: The opposite side is given by u v. u v 2 =

### Chapter 6: Orthogonality

Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

### v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :

Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =

### Linear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products

Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 6A: Inner products In this chapter, the field F = R or C. We regard F equipped with a conjugation χ : F F. If F =

### MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation

### Frames inr n. Brody Dylan Johnson Department of Mathematics Washington University Saint Louis, Missouri

Frames inr n Brody Dylan Johnson Department of Mathematics Washington University Saint Louis, Missouri 63130 e-mail: brody@math.wustl.edu February, 00 Abstract These notes provide an introduction to the

### Math 1180, Notes, 14 1 C. v 1 v n v 2. C A ; w n. A and w = v i w i : v w = i=1

Math 8, 9 Notes, 4 Orthogonality We now start using the dot product a lot. v v = v v n then by Recall that if w w ; w n and w = v w = nx v i w i : Using this denition, we dene the \norm", or length, of

### Linear Algebra. Session 12

Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

### MATH 1120 (LINEAR ALGEBRA 1), FINAL EXAM FALL 2011 SOLUTIONS TO PRACTICE VERSION

MATH (LINEAR ALGEBRA ) FINAL EXAM FALL SOLUTIONS TO PRACTICE VERSION Problem (a) For each matrix below (i) find a basis for its column space (ii) find a basis for its row space (iii) determine whether

### Elementary linear algebra

Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

### Review of Linear Algebra

Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

### 3 a 21 a a 2N. 3 a 21 a a 2M

APPENDIX: MATHEMATICS REVIEW G 12.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers 2 A = 6 4 a 11 a 12... a 1N a 21 a 22... a 2N. 7..... 5 a M1 a M2...

### Quantum Computing Lecture 2. Review of Linear Algebra

Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces

### Math Linear Algebra

Math 220 - Linear Algebra (Summer 208) Solutions to Homework #7 Exercise 6..20 (a) TRUE. u v v u = 0 is equivalent to u v = v u. The latter identity is true due to the commutative property of the inner

### MTH 2032 SemesterII

MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents

### Numerical Linear Algebra

University of Alabama at Birmingham Department of Mathematics Numerical Linear Algebra Lecture Notes for MA 660 (1997 2014) Dr Nikolai Chernov April 2014 Chapter 0 Review of Linear Algebra 0.1 Matrices

### Problem Set 9 Due: In class Tuesday, Nov. 27 Late papers will be accepted until 12:00 on Thursday (at the beginning of class).

Math 3, Fall Jerry L. Kazdan Problem Set 9 Due In class Tuesday, Nov. 7 Late papers will be accepted until on Thursday (at the beginning of class).. Suppose that is an eigenvalue of an n n matrix A and

### Lecture 23: 6.1 Inner Products

Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such

### A 2 G 2 A 1 A 1. (3) A double edge pointing from α i to α j if α i, α j are not perpendicular and α i 2 = 2 α j 2

46 MATH 223A NOTES 2011 LIE ALGEBRAS 11. Classification of semisimple Lie algebras I will explain how the Cartan matrix and Dynkin diagrams describe root systems. Then I will go through the classification

### Mathematical Methods wk 1: Vectors

Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm

### Mathematical Methods wk 1: Vectors

Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm

### Review problems for MA 54, Fall 2004.

Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on

### Math 291-2: Lecture Notes Northwestern University, Winter 2016

Math 291-2: Lecture Notes Northwestern University, Winter 2016 Written by Santiago Cañez These are lecture notes for Math 291-2, the second quarter of MENU: Intensive Linear Algebra and Multivariable Calculus,

### Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures

### MATH 235. Final ANSWERS May 5, 2015

MATH 235 Final ANSWERS May 5, 25. ( points) Fix positive integers m, n and consider the vector space V of all m n matrices with entries in the real numbers R. (a) Find the dimension of V and prove your

### EIGENVALUES AND EIGENVECTORS 3

EIGENVALUES AND EIGENVECTORS 3 1. Motivation 1.1. Diagonal matrices. Perhaps the simplest type of linear transformations are those whose matrix is diagonal (in some basis). Consider for example the matrices

### MATH 431: FIRST MIDTERM. Thursday, October 3, 2013.

MATH 431: FIRST MIDTERM Thursday, October 3, 213. (1) An inner product on the space of matrices. Let V be the vector space of 2 2 real matrices (that is, the algebra Mat 2 (R), but without the mulitiplicative

### GQE ALGEBRA PROBLEMS

GQE ALGEBRA PROBLEMS JAKOB STREIPEL Contents. Eigenthings 2. Norms, Inner Products, Orthogonality, and Such 6 3. Determinants, Inverses, and Linear (In)dependence 4. (Invariant) Subspaces 3 Throughout

### x 3y 2z = 6 1.2) 2x 4y 3z = 8 3x + 6y + 8z = 5 x + 3y 2z + 5t = 4 1.5) 2x + 8y z + 9t = 9 3x + 5y 12z + 17t = 7

Linear Algebra and its Applications-Lab 1 1) Use Gaussian elimination to solve the following systems x 1 + x 2 2x 3 + 4x 4 = 5 1.1) 2x 1 + 2x 2 3x 3 + x 4 = 3 3x 1 + 3x 2 4x 3 2x 4 = 1 x + y + 2z = 4 1.4)

### Review of linear algebra

Review of linear algebra 1 Vectors and matrices We will just touch very briefly on certain aspects of linear algebra, most of which should be familiar. Recall that we deal with vectors, i.e. elements of

### 1 Last time: least-squares problems

MATH Linear algebra (Fall 07) Lecture Last time: least-squares problems Definition. If A is an m n matrix and b R m, then a least-squares solution to the linear system Ax = b is a vector x R n such that

### Linear Algebra Lecture Notes-II

Linear Algebra Lecture Notes-II Vikas Bist Department of Mathematics Panjab University, Chandigarh-64 email: bistvikas@gmail.com Last revised on March 5, 8 This text is based on the lectures delivered

### Vectors in Function Spaces

Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also

### Math 24 Spring 2012 Sample Homework Solutions Week 8

Math 4 Spring Sample Homework Solutions Week 8 Section 5. (.) Test A M (R) for diagonalizability, and if possible find an invertible matrix Q and a diagonal matrix D such that Q AQ = D. ( ) 4 (c) A =.

### MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v

### Supplementary Notes on Linear Algebra

Supplementary Notes on Linear Algebra Mariusz Wodzicki May 3, 2015 1 Vector spaces 1.1 Coordinatization of a vector space 1.1.1 Given a basis B = {b 1,..., b n } in a vector space V, any vector v V can

### Linear Models Review

Linear Models Review Vectors in IR n will be written as ordered n-tuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign

### Spectral Theorems in Euclidean and Hermitian Spaces

Chapter 10 Spectral Theorems in Euclidean and Hermitian Spaces 10.1 Normal Linear Maps Let E be a real Euclidean space (or a complex Hermitian space) with inner product u, v 7! hu, vi. In the real Euclidean

### Inner Product and Orthogonality

Inner Product and Orthogonality P. Sam Johnson October 3, 2014 P. Sam Johnson (NITK) Inner Product and Orthogonality October 3, 2014 1 / 37 Overview In the Euclidean space R 2 and R 3 there are two concepts,

### MATH 22A: LINEAR ALGEBRA Chapter 4

MATH 22A: LINEAR ALGEBRA Chapter 4 Jesús De Loera, UC Davis November 30, 2012 Orthogonality and Least Squares Approximation QUESTION: Suppose Ax = b has no solution!! Then what to do? Can we find an Approximate

### 08a. Operators on Hilbert spaces. 1. Boundedness, continuity, operator norms

(February 24, 2017) 08a. Operators on Hilbert spaces Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ [This document is http://www.math.umn.edu/ garrett/m/real/notes 2016-17/08a-ops

### Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

### Dot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem.

Dot Products K. Behrend April 3, 008 Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem. Contents The dot product 3. Length of a vector........................

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### Diagonalizing Matrices

Diagonalizing Matrices Massoud Malek A A Let A = A k be an n n non-singular matrix and let B = A = [B, B,, B k,, B n ] Then A n A B = A A 0 0 A k [B, B,, B k,, B n ] = 0 0 = I n 0 A n Notice that A i B

### Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank

Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank David Glickenstein November 3, 4 Representing graphs as matrices It will sometimes be useful to represent graphs

### Conceptual Questions for Review

Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.

### if <v;w>=0. The length of a vector v is kvk, its distance from 0. If kvk =1,then v is said to be a unit vector. When V is a real vector space, then on

Function Spaces x1. Inner products and norms. From linear algebra, we recall that an inner product for a complex vector space V is a function < ; >: VV!C that satises the following properties. I1. Positivity:

### SUMMARY OF MATH 1600

SUMMARY OF MATH 1600 Note: The following list is intended as a study guide for the final exam. It is a continuation of the study guide for the midterm. It does not claim to be a comprehensive list. You

### A PRIMER ON SESQUILINEAR FORMS

A PRIMER ON SESQUILINEAR FORMS BRIAN OSSERMAN This is an alternative presentation of most of the material from 8., 8.2, 8.3, 8.4, 8.5 and 8.8 of Artin s book. Any terminology (such as sesquilinear form

### Linear Algebra: Characteristic Value Problem

Linear Algebra: Characteristic Value Problem . The Characteristic Value Problem Let < be the set of real numbers and { be the set of complex numbers. Given an n n real matrix A; does there exist a number

### Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

### CS 246 Review of Linear Algebra 01/17/19

1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

### Finite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product

Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )

### MATH 115A: SAMPLE FINAL SOLUTIONS

MATH A: SAMPLE FINAL SOLUTIONS JOE HUGHES. Let V be the set of all functions f : R R such that f( x) = f(x) for all x R. Show that V is a vector space over R under the usual addition and scalar multiplication

### 2 Garrett: `A Good Spectral Theorem' 1. von Neumann algebras, density theorem The commutant of a subring S of a ring R is S 0 = fr 2 R : rs = sr; 8s 2

1 A Good Spectral Theorem c1996, Paul Garrett, garrett@math.umn.edu version February 12, 1996 1 Measurable Hilbert bundles Measurable Banach bundles Direct integrals of Hilbert spaces Trivializing Hilbert

### (v, w) = arccos( < v, w >

MA322 Sathaye Notes on Inner Products Notes on Chapter 6 Inner product. Given a real vector space V, an inner product is defined to be a bilinear map F : V V R such that the following holds: For all v