Summary of Week 9 B = then A A =

Size: px
Start display at page:

Download "Summary of Week 9 B = then A A ="

Transcription

1 Summary of Week 9 Finding the square root of a positive operator Last time we saw that positive operators have a unique positive square root We now briefly look at how one would go about calculating the square root Notice that the square root of a diagonal matrix is easy to write down If a,, a n are non-negative real numbers and a a B = an then A a a an where we take only positive square roots How do we compute the square root when the matrix of the positive operator in question is not in diagonal form? Since B is a positive operator, we first diagonalize it This is possible since B is self-adjoint and the spectral theorem tells us that B is diagonalizable We do this by finding a basis of eigenvectors of B (which is an equivalent condition of diagonalizability) and forming the matrix P whose columns are these eigenvectors Then P BP = D, a diagonal matrix, whose diagonal entries are exactly the eigenvalues of B This can be re-written as B = P DP We know that the square root of D, since it is a diagonal matrix We have (P DP )(P DP ) = P DP P DP = P DP = B Thus, P DP is the unique positive square root of B Note: We will be soon interested in square roots of A A for any matrix/operator A Although A need not be self-adjoint, A A is always self-adjoint! We have the following analogy between C and L(V ): Polar Decomposition C z z real numbers: z = z nonnegative unit circle: zz = polar form: z = re iθ L(V ) T T self-adjoint: T = T positive isometries: T T = I polar decomposition Polar Decomposition If T L(V ), then there exists an isometry S L(V ) such that T = S T T Proof If T L(V ) and v V, then T v = T v, T v = T T v, v = T T T T v, v = T T v, T T v = T T v and T v = T T v Define S : Im T T ImT by S ( T T v) = T v The idea is to extend Ŝ to an isometry S Im(V ) such that T = S T T We first check that S is well-defined If v, v V satisfy T T v = T T v then T v T v = T (v v ) = T T (v v ) = T T v T T v = 0 which show that T v = T v By definition, S maps Im T T onto Im T, ie, Moreover, S u = u for all u Im T T In particular, S is injective and the rank-nullity theorem implies that dim Im T T = dim Im T Using orthogonal decomposition, since Im T T and Im T are finite dimensional subspaces of V, we have V = Im T T (Im T T )

2 and V = Im T (Im T ) When we have a direct sum, the dimensions add up ie, if V = U U, then dim V = dim U + dim U We saw earlier that dim Im T T = dim Im T, this immediately gives dim(im T T ) = dim(im T ) Hence, orthonormal bases (e,, e m ), (e,, e m) can be chosen for (Im T T ) and (Im T ) respectively Define S : (Im T T ) (Im T ) by S (a e + + a m e m ) = a e + + a m e m Obviously, S w = w for all w (Im T T ) Now let S = S S : Im T T (Im T T ) Im T (Im T ) Then for any v V, S( T T v) = S ( T T v) = T v, showing that T = S T T Now it remains to check that S is an isometry To see this, notice that the Pythagorean theorem implies that if v = u + w V where u Im T T and w (Im T T ) then Hence S is an isometry Sv = S u + S w = S u + S w = u + w = v Singular Value Decomposition Definition Suppose L(V, W ) The singular values of T are defined as the eigenvalues of T T Singular Value Decomposition (SVD)-Matrix version If A M m n (C), then the following decomposition exists: UΣV where Σ is a diagonal matrix with non-negative entries and U and V are unitary matrices, ie, UU = U U = I = V V = V V The decomposition is obtained by finding the matrices U, Σ, V as follows: Remarks: Σ is an m n diagonal matrix with non-negative entries along the diagonal being the singular values of A A: λ λ λ n, where λ i are the eigenvalues of A A V is an n n matrix whose columns form an orthonormal list of eigenvectors w,, w n for A A That is, A Aw i = λ i w i If A has rank r, then the columns u,, u r of A are formed by u j = λj Aw j for j =,, r and u r+,, u m are formed by extending {u,, u r } to an orthonormal basis of R m () Reason for the choice of V, Σ: Observe that A (V Σ U )(UΣV ) = V Σ V, since Σ is a diagonal matrix and hence self-sdjoint and U U = I Since V V = I, this gives V = V so we have Σ as a diagonalization of A A If you haven t already seen this before, we have the following Fact: If A, B, P are matrices such that P BP then the eigenvalues of A coincide with the eigenvalues of B From the above equation, since V = V, the matrices Σ and A A have the same eigenvalues Since Σ is a diagonal matrix with diagonal entries λ i, i =,, n, this immediately gives λ i to be the singular values of A A, using the definition of A A given above This also gives why V is a matrix whose columns form an orthonormal basis of eigenvectors for A A: Since A A is self-adjoint, the spectral theorem ensures the existence of such a basis and

3 equivalently, diagonalizability of A A The diagonalization is, after all done using a matrix whose columns are the eigenvectors of A A The eigenvectors are calculated as usual Since A A is selfadjoint and therefore normal, the eigenvectors of A A are guaranteed to be orthogonal We then normalize them by dividing each eigenvector by its norm to get an orthonormal basis consisting of eigenvectors () Reason for the choice of U: We have the following fact For any A M m n (C), rank(a) = rank(a ) = rank(aa ) = rank(a A) Thus, if A has rank r, then A A also has rank r, which means A A has at most r non-zero eigenvalues Notice that if UΣV, then we also have AV = UΣ, since V V = I If e,, e n denote the column vectors of V and u,, u m denote the column vectors of U, then A[e,, e n ] = [u,, u m ] λ λ λn ie, [Ae,, Ae n ] = [ λ u,, λ m u m ] Now, if i r, then λ i 0 and we have u i = λi Ae i For r < j m, λ j = 0, then Ae j = 0, so u j can be any vector, but we want the matrix U to be unitary, therefore we choose the u j corresponding to j for which λ j = 0 in such a way that they extend u,, u r to an orthonormal basis of R m Solved examples: A Type: Square matrix A with A A having positive eigenvalues Example : Find a singular value decomposition of the matrix ( ) Solution: Step : Find A A: ( ) ( ) = ( ) Step : Find eigenvalues and eigenvectors of A A: To find eigenvalues, we solve the equation det(a A λi) = 0 to get 5 λ λ = 0 This gives us the equation λ 0λ + 6 = 0, solving which, we get λ = 8, λ =, therefore the singular values of A A are λ =, λ = Next, we find eigenvectors by solving the equations (A A λ i I)v i = 0 for i =, 3

4 For λ = 8, we have ( ) [ 3 3 v 3 3 = 0 = v = ] We then normalize it to get e = [ ] For λ =, we have ( ) [ 3 3 v 3 3 = 0 = v = ] We then normalize it to get e = [ ] Step 3: Write down Σ, V : Σ is the diagonal matrix of the same size of A, with singular values along the diagonal This gives us ( ) 0 () Σ = 0 V is the matrix formed by writing the orthonormal eigenvectors of A A as its columns This gives us ( ) / / () V = / / Step 4: Compute U: In this case, since there are no eigenvalues of A A that are equal to zero, the matrix U is given by [u u ] where the column vectors u, u are found using the formula (3) u i = λi Ae i This gives and This gives us u = (4) U = ( ) [ ] [ / / = 0] u = ( ) [ ] / / = ( ) 0 0 [ 0 ] Step 5: Write down the SVD decomposition using equations (), () and (4): ( ) ( ) ( ) UΣV 0 0 = / / 0 0 / / B Type: Square matrix A with A A having a zero eigenvalue: We proceed as before to compute Σ and V The only difference in this case is that we can t find all the column vectors of U using the formula given in equation (3) What we do in this case is that we find the column vectors using the non-zero eigenvalues/singular values and then extend this list to an orthonormal basis of R m In the example below, m =, so at most one eigenvalue of A A is zero If this happens, we get the 4

5 first column vector u using the formula (3) and extend the list to form an orthonormal basis of R : we just pick a vector whose inner product with u [ is zero and then normalize the vector This [ is ] a b straightforward in the case of length two vectors: If is a vector of norm, then the vector b] a is a vector of norm orthogonal to it Note: If r is the rank of the matrix A, we obtain exactly r non-zero eigenvalues in Step If r < m, we find the vectors u,, u r using equation (3) There is no unique choice of vectors that will extend the list u,, u r to an orthonormal basis of R m This won t affect the decomposition, because the zeros in the diagonal of Σ will ensure that the choice of vectors u r+,, u m is irrelevant Let s see this explicitly in the following example: Example : Find the SVD of the matrix ( ) Solution: Following ( ) the same steps as in the previous example for steps to 4: Step : A Step : Eigenvalues of A A are λ = 5, λ = 0 The corresponding orthonormal eigenvectors are e = [ 4, e 5 3] = [ 3 5 4] Step 3: We can now write down Σ, V : ( ) 5 0 Σ =, V = ( ) Step 4: Finding U: Since λ 0, we use equation (3) to find u : u = ( ) ( ) 4 3 4/5 5 = [ ] /5 5 [ ] We then pick u to be 5 in order to make it orthogonal to u and be of norm Thus, U = ( ) 5 Step 5 Finally, we write down the SVD: ( ) 4 3 = ( ) ( ) ( 5 0 4/5 3/ /5 4/5 = ( ) ( ) ( ) 5 0 4/5 3/ /5 4/5 We could simplify even further to get ( ) ( ) ( ) ) 5

6 C Type: A being an m n matrix with m < n: We proceed as before, keeping in mind the different dimensions of U and V and the diagonal matrix Σ Observe that when m < n, the rank of A rank of A m So, although A A is an n n matrix, there will be at least n m eigenvalues that are equal to zero If r is the rank of A, then we will be assured of r positive eigenvalues of A A and while writing the matrix Σ, we shall ignore the (zero) eigenvalues λ m+,, λ n Although, if r < m, the eigenvalues λ r+,, λ m will also be zero, but they shall appear in the diagonal entries of Σ In the example below, m =, n = 3 Example 3: Find the SVD of the matrix ( ) Solution: We have Step : A Step : Eigenvalues of A A are λ =, λ =, λ 3 = 0 The corresponding orthonormal eigenvectors are e = 3, e = 0, e 3 = 3 3 Step 3: We can now write down Σ, V : ( ) 3/ / / 0 0 Σ =, V = / 0 3/ 0 0 3/ / / Step 4: Finding U: Since λ, λ 0, we use equation (3) to find the vectors u, u : u = ( ) 3/ / 3/ = [ ] Thus, u = ( ) / 0 / = [ ] U = ( ) Step 5 Finally, we write down the SVD: ( ) = ( ) ( ) 3/ / 3/ 0 0 / 0 / 0 0 / 3/ / 6

7 D Type: A being an m n matrix with m > n: Example 4: Find the SVD of the matrix Solution: We( have ) 9 9 Step : A 9 9 Step : Eigenvalues of A A are λ = 8, λ = 0 The corresponding orthonormal eigenvectors are e = [ ], e = [ ] Step 3: We can now write down Σ, V : 8 0 Σ = 0 0, V = ( ) 0 0 Step 4: Finding U: Since λ 0, we use equation (3) to find the vectors u : u = ( ) /3 / 8 / = /3 /3 We pick u so that it is orthogonal to u and then normalize it: We choose u = 5 0 Now we need to find a vector u 3 that is orthogonal to both u and u We do this by finding the cross product of u and u However, since we only need orthogonality, we can ignore the normalization for the vectors and find the cross-product of the vectors and to simplify calculations: 0 î ĵ ˆk = î + 4ĵ + 5ˆk 0 Normalizing, we have u 3 = Step 5: Finally, we write down the SVD: ( ) /3 / 5 / 45 = /3 / 5 4/ 8 0 ( ) 45 /3 0 5/ 0 0 / / / /

The Singular Value Decomposition

The Singular Value Decomposition The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will

More information

Notes on singular value decomposition for Math 54. Recall that if A is a symmetric n n matrix, then A has real eigenvalues A = P DP 1 A = P DP T.

Notes on singular value decomposition for Math 54. Recall that if A is a symmetric n n matrix, then A has real eigenvalues A = P DP 1 A = P DP T. Notes on singular value decomposition for Math 54 Recall that if A is a symmetric n n matrix, then A has real eigenvalues λ 1,, λ n (possibly repeated), and R n has an orthonormal basis v 1,, v n, where

More information

Math 113 Final Exam: Solutions

Math 113 Final Exam: Solutions Math 113 Final Exam: Solutions Thursday, June 11, 2013, 3.30-6.30pm. 1. (25 points total) Let P 2 (R) denote the real vector space of polynomials of degree 2. Consider the following inner product on P

More information

Solutions to Final Practice Problems Written by Victoria Kala Last updated 12/5/2015

Solutions to Final Practice Problems Written by Victoria Kala Last updated 12/5/2015 Solutions to Final Practice Problems Written by Victoria Kala vtkala@math.ucsb.edu Last updated /5/05 Answers This page contains answers only. See the following pages for detailed solutions. (. (a x. See

More information

Singular Value Decomposition (SVD) and Polar Form

Singular Value Decomposition (SVD) and Polar Form Chapter 2 Singular Value Decomposition (SVD) and Polar Form 2.1 Polar Form In this chapter, we assume that we are dealing with a real Euclidean space E. Let f: E E be any linear map. In general, it may

More information

be a Householder matrix. Then prove the followings H = I 2 uut Hu = (I 2 uu u T u )u = u 2 uut u

be a Householder matrix. Then prove the followings H = I 2 uut Hu = (I 2 uu u T u )u = u 2 uut u MATH 434/534 Theoretical Assignment 7 Solution Chapter 7 (71) Let H = I 2uuT Hu = u (ii) Hv = v if = 0 be a Householder matrix Then prove the followings H = I 2 uut Hu = (I 2 uu )u = u 2 uut u = u 2u =

More information

Review problems for MA 54, Fall 2004.

Review problems for MA 54, Fall 2004. Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on

More information

University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm

University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm Name: The proctor will let you read the following conditions

More information

UNIT 6: The singular value decomposition.

UNIT 6: The singular value decomposition. UNIT 6: The singular value decomposition. María Barbero Liñán Universidad Carlos III de Madrid Bachelor in Statistics and Business Mathematical methods II 2011-2012 A square matrix is symmetric if A T

More information

Math 312 Final Exam Jerry L. Kazdan May 5, :00 2:00

Math 312 Final Exam Jerry L. Kazdan May 5, :00 2:00 Math 32 Final Exam Jerry L. Kazdan May, 204 2:00 2:00 Directions This exam has three parts. Part A has shorter questions, (6 points each), Part B has 6 True/False questions ( points each), and Part C has

More information

Math 115A: Homework 5

Math 115A: Homework 5 Math 115A: Homework 5 1 Suppose U, V, and W are finite-dimensional vector spaces over a field F, and that are linear a) Prove ker ST ) ker T ) b) Prove nullst ) nullt ) c) Prove imst ) im S T : U V, S

More information

Final Review Written by Victoria Kala SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015

Final Review Written by Victoria Kala SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015 Final Review Written by Victoria Kala vtkala@mathucsbedu SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015 Summary This review contains notes on sections 44 47, 51 53, 61, 62, 65 For your final,

More information

The Spectral Theorem for normal linear maps

The Spectral Theorem for normal linear maps MAT067 University of California, Davis Winter 2007 The Spectral Theorem for normal linear maps Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (March 14, 2007) In this section we come back to the question

More information

Solutions to Final Exam

Solutions to Final Exam Solutions to Final Exam. Let A be a 3 5 matrix. Let b be a nonzero 5-vector. Assume that the nullity of A is. (a) What is the rank of A? 3 (b) Are the rows of A linearly independent? (c) Are the columns

More information

Singular Value Decomposition

Singular Value Decomposition Chapter 5 Singular Value Decomposition We now reach an important Chapter in this course concerned with the Singular Value Decomposition of a matrix A. SVD, as it is commonly referred to, is one of the

More information

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax = . (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)

More information

Contents. Preface for the Instructor. Preface for the Student. xvii. Acknowledgments. 1 Vector Spaces 1 1.A R n and C n 2

Contents. Preface for the Instructor. Preface for the Student. xvii. Acknowledgments. 1 Vector Spaces 1 1.A R n and C n 2 Contents Preface for the Instructor xi Preface for the Student xv Acknowledgments xvii 1 Vector Spaces 1 1.A R n and C n 2 Complex Numbers 2 Lists 5 F n 6 Digression on Fields 10 Exercises 1.A 11 1.B Definition

More information

Jordan Normal Form and Singular Decomposition

Jordan Normal Form and Singular Decomposition University of Debrecen Diagonalization and eigenvalues Diagonalization We have seen that if A is an n n square matrix, then A is diagonalizable if and only if for all λ eigenvalues of A we have dim(u λ

More information

Fall TMA4145 Linear Methods. Exercise set Given the matrix 1 2

Fall TMA4145 Linear Methods. Exercise set Given the matrix 1 2 Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 07 Exercise set Please justify your answers! The most important part is how you arrive at an

More information

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP) MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

More information

The University of Texas at Austin Department of Electrical and Computer Engineering. EE381V: Large Scale Learning Spring 2013.

The University of Texas at Austin Department of Electrical and Computer Engineering. EE381V: Large Scale Learning Spring 2013. The University of Texas at Austin Department of Electrical and Computer Engineering EE381V: Large Scale Learning Spring 2013 Assignment Two Caramanis/Sanghavi Due: Tuesday, Feb. 19, 2013. Computational

More information

DIAGONALIZATION. In order to see the implications of this definition, let us consider the following example Example 1. Consider the matrix

DIAGONALIZATION. In order to see the implications of this definition, let us consider the following example Example 1. Consider the matrix DIAGONALIZATION Definition We say that a matrix A of size n n is diagonalizable if there is a basis of R n consisting of eigenvectors of A ie if there are n linearly independent vectors v v n such that

More information

Spectral Theorem for Self-adjoint Linear Operators

Spectral Theorem for Self-adjoint Linear Operators Notes for the undergraduate lecture by David Adams. (These are the notes I would write if I was teaching a course on this topic. I have included more material than I will cover in the 45 minute lecture;

More information

Singular Value Decomposition

Singular Value Decomposition Singular Value Decomposition Motivatation The diagonalization theorem play a part in many interesting applications. Unfortunately not all matrices can be factored as A = PDP However a factorization A =

More information

The Singular Value Decomposition

The Singular Value Decomposition The Singular Value Decomposition An Important topic in NLA Radu Tiberiu Trîmbiţaş Babeş-Bolyai University February 23, 2009 Radu Tiberiu Trîmbiţaş ( Babeş-Bolyai University)The Singular Value Decomposition

More information

Chapter 7: Symmetric Matrices and Quadratic Forms

Chapter 7: Symmetric Matrices and Quadratic Forms Chapter 7: Symmetric Matrices and Quadratic Forms (Last Updated: December, 06) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved

More information

Math 108b: Notes on the Spectral Theorem

Math 108b: Notes on the Spectral Theorem Math 108b: Notes on the Spectral Theorem From section 6.3, we know that every linear operator T on a finite dimensional inner product space V has an adjoint. (T is defined as the unique linear operator

More information

The Singular Value Decomposition and Least Squares Problems

The Singular Value Decomposition and Least Squares Problems The Singular Value Decomposition and Least Squares Problems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 27, 2009 Applications of SVD solving

More information

Throughout these notes we assume V, W are finite dimensional inner product spaces over C.

Throughout these notes we assume V, W are finite dimensional inner product spaces over C. Math 342 - Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal

More information

Problem # Max points possible Actual score Total 120

Problem # Max points possible Actual score Total 120 FINAL EXAMINATION - MATH 2121, FALL 2017. Name: ID#: Email: Lecture & Tutorial: Problem # Max points possible Actual score 1 15 2 15 3 10 4 15 5 15 6 15 7 10 8 10 9 15 Total 120 You have 180 minutes to

More information

Lecture 19: Isometries, Positive operators, Polar and singular value decompositions; Unitary matrices and classical groups; Previews (1)

Lecture 19: Isometries, Positive operators, Polar and singular value decompositions; Unitary matrices and classical groups; Previews (1) Lecture 19: Isometries, Positive operators, Polar and singular value decompositions; Unitary matrices and classical groups; Previews (1) Travis Schedler Thurs, Nov 18, 2010 (version: Wed, Nov 17, 2:15

More information

MATH 304 Linear Algebra Lecture 34: Review for Test 2.

MATH 304 Linear Algebra Lecture 34: Review for Test 2. MATH 304 Linear Algebra Lecture 34: Review for Test 2. Topics for Test 2 Linear transformations (Leon 4.1 4.3) Matrix transformations Matrix of a linear mapping Similar matrices Orthogonality (Leon 5.1

More information

Definitions for Quizzes

Definitions for Quizzes Definitions for Quizzes Italicized text (or something close to it) will be given to you. Plain text is (an example of) what you should write as a definition. [Bracketed text will not be given, nor does

More information

235 Final exam review questions

235 Final exam review questions 5 Final exam review questions Paul Hacking December 4, 0 () Let A be an n n matrix and T : R n R n, T (x) = Ax the linear transformation with matrix A. What does it mean to say that a vector v R n is an

More information

LinGloss. A glossary of linear algebra

LinGloss. A glossary of linear algebra LinGloss A glossary of linear algebra Contents: Decompositions Types of Matrices Theorems Other objects? Quasi-triangular A matrix A is quasi-triangular iff it is a triangular matrix except its diagonal

More information

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v ) Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

More information

MATRICES ARE SIMILAR TO TRIANGULAR MATRICES

MATRICES ARE SIMILAR TO TRIANGULAR MATRICES MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,

More information

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

LINEAR ALGEBRA 1, 2012-I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS

LINEAR ALGEBRA 1, 2012-I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS LINEAR ALGEBRA, -I PARTIAL EXAM SOLUTIONS TO PRACTICE PROBLEMS Problem (a) For each of the two matrices below, (i) determine whether it is diagonalizable, (ii) determine whether it is orthogonally diagonalizable,

More information

. = V c = V [x]v (5.1) c 1. c k

. = V c = V [x]v (5.1) c 1. c k Chapter 5 Linear Algebra It can be argued that all of linear algebra can be understood using the four fundamental subspaces associated with a matrix Because they form the foundation on which we later work,

More information

Maths for Signals and Systems Linear Algebra in Engineering

Maths for Signals and Systems Linear Algebra in Engineering Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE

More information

Computational Methods. Eigenvalues and Singular Values

Computational Methods. Eigenvalues and Singular Values Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations

More information

LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM

LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM Unless otherwise stated, all vector spaces in this worksheet are finite dimensional and the scalar field F is R or C. Definition 1. A linear operator

More information

The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA)

The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) Chapter 5 The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) 5.1 Basics of SVD 5.1.1 Review of Key Concepts We review some key definitions and results about matrices that will

More information

Linear Algebra Lecture Notes-II

Linear Algebra Lecture Notes-II Linear Algebra Lecture Notes-II Vikas Bist Department of Mathematics Panjab University, Chandigarh-64 email: bistvikas@gmail.com Last revised on March 5, 8 This text is based on the lectures delivered

More information

Final A. Problem Points Score Total 100. Math115A Nadja Hempel 03/23/2017

Final A. Problem Points Score Total 100. Math115A Nadja Hempel 03/23/2017 Final A Math115A Nadja Hempel 03/23/2017 nadja@math.ucla.edu Name: UID: Problem Points Score 1 10 2 20 3 5 4 5 5 9 6 5 7 7 8 13 9 16 10 10 Total 100 1 2 Exercise 1. (10pt) Let T : V V be a linear transformation.

More information

MATH 581D FINAL EXAM Autumn December 12, 2016

MATH 581D FINAL EXAM Autumn December 12, 2016 MATH 58D FINAL EXAM Autumn 206 December 2, 206 NAME: SIGNATURE: Instructions: there are 6 problems on the final. Aim for solving 4 problems, but do as much as you can. Partial credit will be given on all

More information

(VII.E) The Singular Value Decomposition (SVD)

(VII.E) The Singular Value Decomposition (SVD) (VII.E) The Singular Value Decomposition (SVD) In this section we describe a generalization of the Spectral Theorem to non-normal operators, and even to transformations between different vector spaces.

More information

Pseudoinverse & Moore-Penrose Conditions

Pseudoinverse & Moore-Penrose Conditions ECE 275AB Lecture 7 Fall 2008 V1.0 c K. Kreutz-Delgado, UC San Diego p. 1/1 Lecture 7 ECE 275A Pseudoinverse & Moore-Penrose Conditions ECE 275AB Lecture 7 Fall 2008 V1.0 c K. Kreutz-Delgado, UC San Diego

More information

1. The Polar Decomposition

1. The Polar Decomposition A PERSONAL INTERVIEW WITH THE SINGULAR VALUE DECOMPOSITION MATAN GAVISH Part. Theory. The Polar Decomposition In what follows, F denotes either R or C. The vector space F n is an inner product space with

More information

Dimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas

Dimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx

More information

Recall the convention that, for us, all vectors are column vectors.

Recall the convention that, for us, all vectors are column vectors. Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

More information

Dimension. Eigenvalue and eigenvector

Dimension. Eigenvalue and eigenvector Dimension. Eigenvalue and eigenvector Math 112, week 9 Goals: Bases, dimension, rank-nullity theorem. Eigenvalue and eigenvector. Suggested Textbook Readings: Sections 4.5, 4.6, 5.1, 5.2 Week 9: Dimension,

More information

Math Homework 8 (selected problems)

Math Homework 8 (selected problems) Math 102 - Homework 8 (selected problems) David Lipshutz Problem 1. (Strang, 5.5: #14) In the list below, which classes of matrices contain A and which contain B? 1 1 1 1 A 0 0 1 0 0 0 0 1 and B 1 1 1

More information

Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam

Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system

More information

MATH 221, Spring Homework 10 Solutions

MATH 221, Spring Homework 10 Solutions MATH 22, Spring 28 - Homework Solutions Due Tuesday, May Section 52 Page 279, Problem 2: 4 λ A λi = and the characteristic polynomial is det(a λi) = ( 4 λ)( λ) ( )(6) = λ 6 λ 2 +λ+2 The solutions to the

More information

Lecture 7: Positive Semidefinite Matrices

Lecture 7: Positive Semidefinite Matrices Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.

More information

ANSWERS (5 points) Let A be a 2 2 matrix such that A =. Compute A. 2

ANSWERS (5 points) Let A be a 2 2 matrix such that A =. Compute A. 2 MATH 7- Final Exam Sample Problems Spring 7 ANSWERS ) ) ). 5 points) Let A be a matrix such that A =. Compute A. ) A = A ) = ) = ). 5 points) State ) the definition of norm, ) the Cauchy-Schwartz inequality

More information

1. Select the unique answer (choice) for each problem. Write only the answer.

1. Select the unique answer (choice) for each problem. Write only the answer. MATH 5 Practice Problem Set Spring 7. Select the unique answer (choice) for each problem. Write only the answer. () Determine all the values of a for which the system has infinitely many solutions: x +

More information

Linear Algebra, part 3. Going back to least squares. Mathematical Models, Analysis and Simulation = 0. a T 1 e. a T n e. Anna-Karin Tornberg

Linear Algebra, part 3. Going back to least squares. Mathematical Models, Analysis and Simulation = 0. a T 1 e. a T n e. Anna-Karin Tornberg Linear Algebra, part 3 Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2010 Going back to least squares (Sections 1.7 and 2.3 from Strang). We know from before: The vector

More information

Lecture 12: Diagonalization

Lecture 12: Diagonalization Lecture : Diagonalization A square matrix D is called diagonal if all but diagonal entries are zero: a a D a n 5 n n. () Diagonal matrices are the simplest matrices that are basically equivalent to vectors

More information

Math 4153 Exam 3 Review. The syllabus for Exam 3 is Chapter 6 (pages ), Chapter 7 through page 137, and Chapter 8 through page 182 in Axler.

Math 4153 Exam 3 Review. The syllabus for Exam 3 is Chapter 6 (pages ), Chapter 7 through page 137, and Chapter 8 through page 182 in Axler. Math 453 Exam 3 Review The syllabus for Exam 3 is Chapter 6 (pages -2), Chapter 7 through page 37, and Chapter 8 through page 82 in Axler.. You should be sure to know precise definition of the terms we

More information

Computational math: Assignment 1

Computational math: Assignment 1 Computational math: Assignment 1 Thanks Ting Gao for her Latex file 11 Let B be a 4 4 matrix to which we apply the following operations: 1double column 1, halve row 3, 3add row 3 to row 1, 4interchange

More information

7. Symmetric Matrices and Quadratic Forms

7. Symmetric Matrices and Quadratic Forms Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value

More information

Linear Algebra using Dirac Notation: Pt. 2

Linear Algebra using Dirac Notation: Pt. 2 Linear Algebra using Dirac Notation: Pt. 2 PHYS 476Q - Southern Illinois University February 6, 2018 PHYS 476Q - Southern Illinois University Linear Algebra using Dirac Notation: Pt. 2 February 6, 2018

More information

The following definition is fundamental.

The following definition is fundamental. 1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic

More information

MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, Chapter 1: Linear Equations and Matrices MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

More information

Last name: First name: Signature: Student number:

Last name: First name: Signature: Student number: MAT 2141 The final exam Instructor: K. Zaynullin Last name: First name: Signature: Student number: Do not detach the pages of this examination. You may use the back of the pages as scrap paper for calculations,

More information

18.06SC Final Exam Solutions

18.06SC Final Exam Solutions 18.06SC Final Exam Solutions 1 (4+7=11 pts.) Suppose A is 3 by 4, and Ax = 0 has exactly 2 special solutions: 1 2 x 1 = 1 and x 2 = 1 1 0 0 1 (a) Remembering that A is 3 by 4, find its row reduced echelon

More information

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2017 LECTURE 5

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2017 LECTURE 5 STAT 39: MATHEMATICAL COMPUTATIONS I FALL 17 LECTURE 5 1 existence of svd Theorem 1 (Existence of SVD) Every matrix has a singular value decomposition (condensed version) Proof Let A C m n and for simplicity

More information

6 Inner Product Spaces

6 Inner Product Spaces Lectures 16,17,18 6 Inner Product Spaces 6.1 Basic Definition Parallelogram law, the ability to measure angle between two vectors and in particular, the concept of perpendicularity make the euclidean space

More information

Final Exam, Linear Algebra, Fall, 2003, W. Stephen Wilson

Final Exam, Linear Algebra, Fall, 2003, W. Stephen Wilson Final Exam, Linear Algebra, Fall, 2003, W. Stephen Wilson Name: TA Name and section: NO CALCULATORS, SHOW ALL WORK, NO OTHER PAPERS ON DESK. There is very little actual work to be done on this exam if

More information

The Singular Value Decomposition

The Singular Value Decomposition CHAPTER 6 The Singular Value Decomposition Exercise 67: SVD examples (a) For A =[, 4] T we find a matrixa T A =5,whichhastheeigenvalue =5 Thisprovidesuswiththesingularvalue =+ p =5forA Hence the matrix

More information

Remark 1 By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.

Remark 1 By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero. Sec 5 Eigenvectors and Eigenvalues In this chapter, vector means column vector Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called

More information

Math 291-2: Lecture Notes Northwestern University, Winter 2016

Math 291-2: Lecture Notes Northwestern University, Winter 2016 Math 291-2: Lecture Notes Northwestern University, Winter 2016 Written by Santiago Cañez These are lecture notes for Math 291-2, the second quarter of MENU: Intensive Linear Algebra and Multivariable Calculus,

More information

Linear Least Squares. Using SVD Decomposition.

Linear Least Squares. Using SVD Decomposition. Linear Least Squares. Using SVD Decomposition. Dmitriy Leykekhman Spring 2011 Goals SVD-decomposition. Solving LLS with SVD-decomposition. D. Leykekhman Linear Least Squares 1 SVD Decomposition. For any

More information

Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

More information

Lecture 10 - Eigenvalues problem

Lecture 10 - Eigenvalues problem Lecture 10 - Eigenvalues problem Department of Computer Science University of Houston February 28, 2008 1 Lecture 10 - Eigenvalues problem Introduction Eigenvalue problems form an important class of problems

More information

Review of some mathematical tools

Review of some mathematical tools MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical

More information

Midterm 2 Solutions, MATH 54, Linear Algebra and Differential Equations, Fall 2014

Midterm 2 Solutions, MATH 54, Linear Algebra and Differential Equations, Fall 2014 Name (Last, First): Student ID: Circle your section: 2 Shin 8am 7 Evans 22 Lim pm 35 Etcheverry 22 Cho 8am 75 Evans 23 Tanzer 2pm 35 Evans 23 Shin 9am 5 Latimer 24 Moody 2pm 8 Evans 24 Cho 9am 254 Sutardja

More information

Math 414: Linear Algebra II, Fall 2015 Final Exam

Math 414: Linear Algebra II, Fall 2015 Final Exam Math 414: Linear Algebra II, Fall 2015 Final Exam December 17, 2015 Name: This is a closed book single author exam. Use of books, notes, or other aids is not permissible, nor is collaboration with any

More information

Linear Algebra. Session 12

Linear Algebra. Session 12 Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

More information

Unit 5: Matrix diagonalization

Unit 5: Matrix diagonalization Unit 5: Matrix diagonalization Juan Luis Melero and Eduardo Eyras October 2018 1 Contents 1 Matrix diagonalization 3 1.1 Definitions............................. 3 1.1.1 Similar matrix.......................

More information

CS 143 Linear Algebra Review

CS 143 Linear Algebra Review CS 143 Linear Algebra Review Stefan Roth September 29, 2003 Introductory Remarks This review does not aim at mathematical rigor very much, but instead at ease of understanding and conciseness. Please see

More information

Designing Information Devices and Systems II

Designing Information Devices and Systems II EECS 16B Fall 2016 Designing Information Devices and Systems II Linear Algebra Notes Introduction In this set of notes, we will derive the linear least squares equation, study the properties symmetric

More information

DEPARTMENT OF MATHEMATICS

DEPARTMENT OF MATHEMATICS Name(Last/First): GUID: DEPARTMENT OF MATHEMATICS Ma322005(Sathaye) - Final Exam Spring 2017 May 3, 2017 DO NOT TURN THIS PAGE UNTIL YOU ARE INSTRUCTED TO DO SO. Be sure to show all work and justify your

More information

ANSWERS. E k E 2 E 1 A = B

ANSWERS. E k E 2 E 1 A = B MATH 7- Final Exam Spring ANSWERS Essay Questions points Define an Elementary Matrix Display the fundamental matrix multiply equation which summarizes a sequence of swap, combination and multiply operations,

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

linearly indepedent eigenvectors as the multiplicity of the root, but in general there may be no more than one. For further discussion, assume matrice

linearly indepedent eigenvectors as the multiplicity of the root, but in general there may be no more than one. For further discussion, assume matrice 3. Eigenvalues and Eigenvectors, Spectral Representation 3.. Eigenvalues and Eigenvectors A vector ' is eigenvector of a matrix K, if K' is parallel to ' and ' 6, i.e., K' k' k is the eigenvalue. If is

More information

Then since v is an eigenvector of T, we have (T λi)v = 0. Then

Then since v is an eigenvector of T, we have (T λi)v = 0. Then Problem F02.10. Let T be a linear operator on a finite dimensional complex inner product space V such that T T = T T. Show that there is an orthonormal basis of V consisting of eigenvectors of B. Solution.

More information

MIT Final Exam Solutions, Spring 2017

MIT Final Exam Solutions, Spring 2017 MIT 8.6 Final Exam Solutions, Spring 7 Problem : For some real matrix A, the following vectors form a basis for its column space and null space: C(A) = span,, N(A) = span,,. (a) What is the size m n of

More information

LINEAR ALGEBRA SUMMARY SHEET.

LINEAR ALGEBRA SUMMARY SHEET. LINEAR ALGEBRA SUMMARY SHEET RADON ROSBOROUGH https://intuitiveexplanationscom/linear-algebra-summary-sheet/ This document is a concise collection of many of the important theorems of linear algebra, organized

More information

Singular value decomposition

Singular value decomposition Singular value decomposition The eigenvalue decomposition (EVD) for a square matrix A gives AU = UD. Let A be rectangular (m n, m > n). A singular value σ and corresponding pair of singular vectors u (m

More information

Remark By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.

Remark By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero. Sec 6 Eigenvalues and Eigenvectors Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called an eigenvalue of A if there is a nontrivial

More information

Homework 9. Ha Pham. December 6, 2008

Homework 9. Ha Pham. December 6, 2008 Homework 9 Ha Pham December 6, 2008 Problem (Ch7 - Problem 30). Suppose S L(V ). Prove that S is an isometry if and only if all the singular values of S equal. Proof. S S is self-adjoint operator with

More information

Linear Algebra - Part II

Linear Algebra - Part II Linear Algebra - Part II Projection, Eigendecomposition, SVD (Adapted from Sargur Srihari s slides) Brief Review from Part 1 Symmetric Matrix: A = A T Orthogonal Matrix: A T A = AA T = I and A 1 = A T

More information

Linear Algebra Review. Fei-Fei Li

Linear Algebra Review. Fei-Fei Li Linear Algebra Review Fei-Fei Li 1 / 37 Vectors Vectors and matrices are just collections of ordered numbers that represent something: movements in space, scaling factors, pixel brightnesses, etc. A vector

More information

TBP MATH33A Review Sheet. November 24, 2018

TBP MATH33A Review Sheet. November 24, 2018 TBP MATH33A Review Sheet November 24, 2018 General Transformation Matrices: Function Scaling by k Orthogonal projection onto line L Implementation If we want to scale I 2 by k, we use the following: [

More information

MATH36001 Generalized Inverses and the SVD 2015

MATH36001 Generalized Inverses and the SVD 2015 MATH36001 Generalized Inverses and the SVD 201 1 Generalized Inverses of Matrices A matrix has an inverse only if it is square and nonsingular. However there are theoretical and practical applications

More information