HW #3 Solutions: M552 Spring (c)

Size: px
Start display at page:

Download "HW #3 Solutions: M552 Spring (c)"

Transcription

1 HW #3 Solutions: M55 Spring 6. (4.-Trefethen & Bau: parts (a), (c) and (e)) Determine the SVDs of the following matrices (by hand calculation): (a) [ 3 (c) (e) [ ANS: In each case we seek A UΣV. The general algorithm is to first find A A (UΣV ) UΣV V Σ ΣV V DV, the spectral decomposition of the hermitian matrix A A. Then take Σ D, noting that Σ may have to be extended to the size of A, i.e. zero rows appended, and solve for U from AV UΣ. However, if A A, then we can use A s spectral decompostion A PDP P D sign(d)p UΣV, where U P, Σ D and V sign(d)p. (a): A A, and in fact A is already in diagonal form. So D A and P I. Then [ [ [ A PDP 3 IDI I D sign(d)i UΣV. [ [ (c): Following the general algorithm, A A I I 4 4, the spectral decompostion of A A. Note that the diagonal entries of D are not decreasing. We can achieve this by interchanging the columns of I and the diagonal entries of D, which simply amounts to reordering the e-pairs of [ [ [ [ [ [ 4 4 A A, giving A A. Then extending, 4 [ Σ and V. Now choose U unitary so that AV UΣ, or [u u u 3. We see that this gives U I. Thus, A [ UΣV. (e): A A, and it is easy to see that [ the e-values of A [ are λ and λ, and corresponding / mutually orthogonal e-vectors p / / and p / (do the computation if you don t see this). Then A PDP [ / / / / [ [ / / / / UΣV.

2 . (4.3-Trefethen & Bau) Write a MATLAB program which, given a real matrix A, plots the right singular vectors v and v in the unit circle and also the left singular vectors[ u and u in the appropriate ellipse, as in Figure 4.. Apply your program to the matrix (3.7), A, and also the matrices of exercise 4.. You may use MATLAB s svd command to generate the singular vectors and values. ANS: Here is the code for a given matrix A: [U,S,Vsvd(A); Sdiag(S); % find SVD th:*pi/56:*pi; dom[cos(th); sin(th); rana*dom; % define points on unit circle and image subplot(,,) plot(dom(,:),dom(,:),. ),axis( image ),grid,hold arrow([,,[v(,),v(,)) arrow([,,[v(,),v(,)) title( Right Singular Vectors ) subplot(,,) plot(ran(,:),ran(,:), r. ),axis( image ),grid,hold arrow([,,s()*[u(,),u(,)) if (abs(s()) > e-) % check that singular value is nonzero arrow([,,s()*[u(,),u(,)) end title( Image & Left Singular Vector(s) ) % arrow.m is a user-contributed M-file available at Results for A [ : Right Singular Vectors Image & Left Singular Vector(s)

3 [ 3 Results for A :.5 Right Singular Vectors Image & Left Singular Vector(s) Results for A [ 3 : 3 Image & Left Singular Vector(s) Right Singular Vectors

4 [ Results for A : Right Singular Vectors Image & Left Singular Vector(s) Results for A [ : Right Singular Vectors Image & Left Singular Vector(s)

5 3. (4.5-Trefethen & Bau) Theorem 4. asserts that every A C m n has an SVD A UΣV. Show that is A is real, then it has a real SVD (U R m m,v R n n ). ANS: If A R m n then A A A T A R n n is symmetric, hence A T A V DV T where D R n n is a real diagonal matrix, with non-negative entries (we proved this in class), and V R n n is a real orthogonal matrix. Hence Σ D R n n is defined. If m > n we can append zero rows to Σ to ensure Σ R m n. What if m < n? Think about it! We can solve for U using AV UΣ, showing that U can be chosen to ensure U R m m since A,Σ and V are all real matrices.

6 [ 4. (5.-Trefethen & Bau) In Example 3. we considered the matrix (3.7), A, and asserted, among other things, that its -norm is approximately.98. Using the SVD, work out (on paper) the exact values of σ min (A) and σ max (A) for this matrix. ANS: To find the singular values, form A A (UΣV ) UΣV V Σ ΣV V DV, the spectral decomposition of the hermitian matrix A A. The e-values are the diagonal entries of D, and Σ D. [ [ [ [ A A A A λi λ 8 λ 9λ + 4, 8 λ whose roots are λ, 9 ± ( 9) 4 4 σ min (A) giving A σ max (A) ± 65. So we have , σ max (A) ,

7 5. (5.4-Trefethen & Bau) Suppose A C m m has an [ SVD A UΣV. Find an eigenvalue decomposition A (5.), XΛX, of the m m hermitian matrix. A ANS: Working on it!

8 6. Find the eigenvalues and a corresponding eigenfunction for the operator d /dx applied to functions with homogeneous boundary conditions on [,, i.e., solutions of y λy, y() y(). Note that there are a infinite number of them, say (λ k,y k (x)) for k,,..., with < λ < λ <... ANS: We have to solve a second order constant coefficient ODE, whose solution must also satisfy the given boundary conditions (BCs). We have y + λy r + λ r, ± λ. The specific form of the solution depends on the sign of λ as follows: λ < : So λ >. The general solution is given by y(x) C e λx + C e λx. Applying the BCs, or in matrix form [ e λ e λ [ C C y() C e λ + C e λ C + C, y() C e λ + C e λ C e λ + C e λ, [ C C since [ e λ e λ so the system has only the trivial solution, thus y(x) which can t be an eigenfunction. e λ e λ, λ : The general solution is given by y(x) C + C x. Applying the BCs, y() C + C C C, y() C C C. So again, we arrive at the trivial solution y(x) which can t be an eigenfunction. λ > : So λ < which gives rise to complex roots r,. In this case the general real solution is given by y(x) C cos λx + C sin λx. Applying the BCs, y() C cos λ + C sin λ C + C C C, y() C sin λ C sin λ. Choosing C gives y(x), the trivial solution. For a nontrivial solution we must have sin λ, which is satisfied by an infinite number of λ >, given by (note λ > ) λk kπ for k,,..., λ k (kπ) for k,,.... These {λ k } k are thus the eigenvalues of the continuous second derivative operator for functions defined on [, with homogeneous BCs. For each k, taking C, a corresponding eigenfunction is given by y k (x) sin λ k x sin kπx. Summarizing, the e-pairs are given by (λ k,y k (x)) ((kπ),sin kπx) for k,,..., and note < λ < λ <... with lim k λ k.

9 7. Consider the discrete analog of the eigenproblem in the previous problem, with d /dx D + D D. To this end, choose N >, let h /N and x i i h for i,,...,n. Note we now have N + grid points with x and x N. So we seek e-pairs which satisfy or in matrix form, ( D u) i u i + u i u i+ h λu i for i,,...,n, u u N, h u u. u N u N λ u u. u N u N (a) Show that (λ k,u k ) is an e-pair for k,,...,n where λ k ( cos kπh)/h and u k is the vector with components (u k ) i sin ikπh. Hint: Use trig identities, and note that sin kπh sin Nkπh. (b) For each N 8 and N 6 plot graphs (using subplot) of the continuous eigenfunctions found in the previous problem, and the discrete eigenvectors from (a) for k,, N/ and N. Thus each plot should have 4 images. Use a linetype for the continuous eigenfunctions and symbols for the discrete eigenvectors. Label and title your graphs. (c) In a single plot display the first 3 eigenvalues vs. there number, i.e., k,,...,3 of the continuous problem along with the N eigenvalues for each N 8,6 and 3. For the continuous case plot the eigenvalues using a linetype with a symbol, and in the discrete case just symbols. Label, title, and place a legend on your plot. Discuss the results. ANS: (a) Noting that sin kπh sin Nkπh. Fix a k, k,...,n. For i,...,n, u k,i + u k,i u k,i+ h sin (i )kπh + sin ikπh sin (i + )kπh h (sin ikπhcos kπh cos ikπhsinkπh)+ sinikπh (sin ikπhcos kπh+cos ikπhsin kπh) h sin ikπh cos kπh + sin ikπh h ( cos kπh) sin ikπh h λ k sin ikπh. λ k u k,i. So we have Au k λ k u k for k,...,n, where A is the matrix representing D. Thus, (λ k,u k ) are e-pairs for k,...,n.

10 (b) Here is the code for N 8: N8; h/n; x:h:; xx:.:; k[ N/ N-; for i:4 subplot(,,i) plot(xx,sin(k(i)*pi*xx),x,sin(k(i)*pi*x), o ) grid,axis( tight ) title([ sin(,numstr(k(i)), \pix): N,numstr(N),, k,numstr(k(i))); end And the graphs for N 8 and N 6: sin(πx): N8, k sin(πx): N8, k sin(4πx): N8, k4 sin(7πx): N8, k sin(πx): N6, k sin(πx): N6, k sin(8πx): N6, k8 sin(5πx): N6, k

11 (c) Here is the code to generate the plot, and the plot itself: N[8 6 3; h./n; NN-; plot(:3,(pi*(:3)).^, -o,... :N(),*(-cos((:N())*pi*h()))/(h()^), *,... :N(),*(-cos((:N())*pi*h()))/(h()^), +,... :N(3),*(-cos((:N(3))*pi*h(3)))/(h(3)^), p ) grid,title( Continuous vs. Discrete spectrum of -d^/dx^ ) ylabel( eigenvalue ),legend( continuous, N8, N6, N3,) continuous N8 N6 N3 Continuous vs. Discrete spectrum of d /dx 8 eigenvalue Discussion: From part (b) we see that the eigenvectors of D are the corresponding eigenfunctions of the continuous problem evaluated at the grid points determined by the specific N. In this sense the discrete problem is clearly a very good approximation to the continuous one. On the other hand, note in the plot above that while for each N the first few discrete eigenvalues are good approximations, this is not the case for the eigenvalues corresponding to the higher modes. Thus one expects these modes in the continuous problems not to be modeled well by the discrete approximation. Note: A Taylor series expansion gives (kπ) λ k (kπ) h( cos kπh) (kπ) (kπh) h( ( + (kπh)4 )) 4 (kπ) ( (kπh) ) h (kπh)4 + (kπ) (kπ) + h ( (kπh) 4 ( ) 4 4 (kπ) 4 h O(h 4 ) O(h ) ) So we do have the expected accuracy of approximation, but remember there is still a term in the error that depends on the eigenvalue being approximated, (kπ) 4 /, which explains, even for larger N, why we do not see rapid convergence of the discrete eigenvalues to the continuous ones as k N. In fact, for a fixed N and k large, we see divergence in the upper spectrum.

MATH 552 Spectral Methods Spring Homework Set 5 - SOLUTIONS

MATH 552 Spectral Methods Spring Homework Set 5 - SOLUTIONS MATH 55 Spectral Methods Spring 9 Homework Set 5 - SOLUTIONS. Suppose you are given an n n linear system Ax = f where the matrix A is tridiagonal b c a b c. A =.........,. a n b n c n a n b n with x =

More information

Notes on Eigenvalues, Singular Values and QR

Notes on Eigenvalues, Singular Values and QR Notes on Eigenvalues, Singular Values and QR Michael Overton, Numerical Computing, Spring 2017 March 30, 2017 1 Eigenvalues Everyone who has studied linear algebra knows the definition: given a square

More information

UNIT 6: The singular value decomposition.

UNIT 6: The singular value decomposition. UNIT 6: The singular value decomposition. María Barbero Liñán Universidad Carlos III de Madrid Bachelor in Statistics and Business Mathematical methods II 2011-2012 A square matrix is symmetric if A T

More information

The Singular Value Decomposition

The Singular Value Decomposition The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will

More information

HW #1 Solutions: M552 Spring 2006

HW #1 Solutions: M552 Spring 2006 HW #1 Solutions: M55 Spring 006 1. (1.4-Trefethen & Bau) Let f 1,...,f 8 be a set of functions defined on the interval [1,8] with the property that for any numbers d 1,...,d 8, there exists a set of coefficients

More information

Linear Algebra Review. Fei-Fei Li

Linear Algebra Review. Fei-Fei Li Linear Algebra Review Fei-Fei Li 1 / 37 Vectors Vectors and matrices are just collections of ordered numbers that represent something: movements in space, scaling factors, pixel brightnesses, etc. A vector

More information

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax = . (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)

More information

Homework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I)

Homework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I) CS 106 Spring 2004 Homework 1 Elena Davidson 8 April 2004 Problem 1.1 Let B be a 4 4 matrix to which we apply the following operations: 1. double column 1, 2. halve row 3, 3. add row 3 to row 1, 4. interchange

More information

Linear Algebra, part 3 QR and SVD

Linear Algebra, part 3 QR and SVD Linear Algebra, part 3 QR and SVD Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2012 Going back to least squares (Section 1.4 from Strang, now also see section 5.2). We

More information

Computational math: Assignment 1

Computational math: Assignment 1 Computational math: Assignment 1 Thanks Ting Gao for her Latex file 11 Let B be a 4 4 matrix to which we apply the following operations: 1double column 1, halve row 3, 3add row 3 to row 1, 4interchange

More information

COMP 558 lecture 18 Nov. 15, 2010

COMP 558 lecture 18 Nov. 15, 2010 Least squares We have seen several least squares problems thus far, and we will see more in the upcoming lectures. For this reason it is good to have a more general picture of these problems and how to

More information

Linear Algebra, part 3. Going back to least squares. Mathematical Models, Analysis and Simulation = 0. a T 1 e. a T n e. Anna-Karin Tornberg

Linear Algebra, part 3. Going back to least squares. Mathematical Models, Analysis and Simulation = 0. a T 1 e. a T n e. Anna-Karin Tornberg Linear Algebra, part 3 Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2010 Going back to least squares (Sections 1.7 and 2.3 from Strang). We know from before: The vector

More information

Singular Value Decomposition

Singular Value Decomposition Singular Value Decomposition Motivatation The diagonalization theorem play a part in many interesting applications. Unfortunately not all matrices can be factored as A = PDP However a factorization A =

More information

Chapter 7: Symmetric Matrices and Quadratic Forms

Chapter 7: Symmetric Matrices and Quadratic Forms Chapter 7: Symmetric Matrices and Quadratic Forms (Last Updated: December, 06) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved

More information

Properties of Linear Transformations from R n to R m

Properties of Linear Transformations from R n to R m Properties of Linear Transformations from R n to R m MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Topic Overview Relationship between the properties of a matrix transformation

More information

Fall TMA4145 Linear Methods. Exercise set Given the matrix 1 2

Fall TMA4145 Linear Methods. Exercise set Given the matrix 1 2 Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 07 Exercise set Please justify your answers! The most important part is how you arrive at an

More information

Notes on Linear Algebra

Notes on Linear Algebra 1 Notes on Linear Algebra Jean Walrand August 2005 I INTRODUCTION Linear Algebra is the theory of linear transformations Applications abound in estimation control and Markov chains You should be familiar

More information

CS 143 Linear Algebra Review

CS 143 Linear Algebra Review CS 143 Linear Algebra Review Stefan Roth September 29, 2003 Introductory Remarks This review does not aim at mathematical rigor very much, but instead at ease of understanding and conciseness. Please see

More information

be a Householder matrix. Then prove the followings H = I 2 uut Hu = (I 2 uu u T u )u = u 2 uut u

be a Householder matrix. Then prove the followings H = I 2 uut Hu = (I 2 uu u T u )u = u 2 uut u MATH 434/534 Theoretical Assignment 7 Solution Chapter 7 (71) Let H = I 2uuT Hu = u (ii) Hv = v if = 0 be a Householder matrix Then prove the followings H = I 2 uut Hu = (I 2 uu )u = u 2 uut u = u 2u =

More information

MATH 612 Computational methods for equation solving and function minimization Week # 2

MATH 612 Computational methods for equation solving and function minimization Week # 2 MATH 612 Computational methods for equation solving and function minimization Week # 2 Instructor: Francisco-Javier Pancho Sayas Spring 2014 University of Delaware FJS MATH 612 1 / 38 Plan for this week

More information

Study Guide for Linear Algebra Exam 2

Study Guide for Linear Algebra Exam 2 Study Guide for Linear Algebra Exam 2 Term Vector Space Definition A Vector Space is a nonempty set V of objects, on which are defined two operations, called addition and multiplication by scalars (real

More information

Notes on singular value decomposition for Math 54. Recall that if A is a symmetric n n matrix, then A has real eigenvalues A = P DP 1 A = P DP T.

Notes on singular value decomposition for Math 54. Recall that if A is a symmetric n n matrix, then A has real eigenvalues A = P DP 1 A = P DP T. Notes on singular value decomposition for Math 54 Recall that if A is a symmetric n n matrix, then A has real eigenvalues λ 1,, λ n (possibly repeated), and R n has an orthonormal basis v 1,, v n, where

More information

Preliminary Examination, Numerical Analysis, August 2016

Preliminary Examination, Numerical Analysis, August 2016 Preliminary Examination, Numerical Analysis, August 2016 Instructions: This exam is closed books and notes. The time allowed is three hours and you need to work on any three out of questions 1-4 and any

More information

Singular Value Decomposition

Singular Value Decomposition Chapter 5 Singular Value Decomposition We now reach an important Chapter in this course concerned with the Singular Value Decomposition of a matrix A. SVD, as it is commonly referred to, is one of the

More information

Maths for Signals and Systems Linear Algebra in Engineering

Maths for Signals and Systems Linear Algebra in Engineering Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE

More information

The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA)

The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) Chapter 5 The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) 5.1 Basics of SVD 5.1.1 Review of Key Concepts We review some key definitions and results about matrices that will

More information

MIT Final Exam Solutions, Spring 2017

MIT Final Exam Solutions, Spring 2017 MIT 8.6 Final Exam Solutions, Spring 7 Problem : For some real matrix A, the following vectors form a basis for its column space and null space: C(A) = span,, N(A) = span,,. (a) What is the size m n of

More information

Linear Algebra Review. Fei-Fei Li

Linear Algebra Review. Fei-Fei Li Linear Algebra Review Fei-Fei Li 1 / 51 Vectors Vectors and matrices are just collections of ordered numbers that represent something: movements in space, scaling factors, pixel brightnesses, etc. A vector

More information

18.06 Problem Set 9 - Solutions Due Wednesday, 21 November 2007 at 4 pm in

18.06 Problem Set 9 - Solutions Due Wednesday, 21 November 2007 at 4 pm in 8.6 Problem Set 9 - Solutions Due Wednesday, 2 November 27 at 4 pm in 2-6. Problem : (5) When A = SΛS is a real-symmetric (or Hermitian) matrix, its eigenvectors can be chosen orthonormal and hence S =

More information

The Singular Value Decomposition and Least Squares Problems

The Singular Value Decomposition and Least Squares Problems The Singular Value Decomposition and Least Squares Problems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 27, 2009 Applications of SVD solving

More information

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2017 LECTURE 5

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2017 LECTURE 5 STAT 39: MATHEMATICAL COMPUTATIONS I FALL 17 LECTURE 5 1 existence of svd Theorem 1 (Existence of SVD) Every matrix has a singular value decomposition (condensed version) Proof Let A C m n and for simplicity

More information

Linear Algebra. Session 12

Linear Algebra. Session 12 Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

More information

Review problems for MA 54, Fall 2004.

Review problems for MA 54, Fall 2004. Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

Numerical Linear Algebra Homework Assignment - Week 2

Numerical Linear Algebra Homework Assignment - Week 2 Numerical Linear Algebra Homework Assignment - Week 2 Đoàn Trần Nguyên Tùng Student ID: 1411352 8th October 2016 Exercise 2.1: Show that if a matrix A is both triangular and unitary, then it is diagonal.

More information

Linear Least Squares. Using SVD Decomposition.

Linear Least Squares. Using SVD Decomposition. Linear Least Squares. Using SVD Decomposition. Dmitriy Leykekhman Spring 2011 Goals SVD-decomposition. Solving LLS with SVD-decomposition. D. Leykekhman Linear Least Squares 1 SVD Decomposition. For any

More information

Functional Analysis Review

Functional Analysis Review Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all

More information

7. Symmetric Matrices and Quadratic Forms

7. Symmetric Matrices and Quadratic Forms Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value

More information

18.06 Problem Set 10 - Solutions Due Thursday, 29 November 2007 at 4 pm in

18.06 Problem Set 10 - Solutions Due Thursday, 29 November 2007 at 4 pm in 86 Problem Set - Solutions Due Thursday, 29 November 27 at 4 pm in 2-6 Problem : (5=5+5+5) Take any matrix A of the form A = B H CB, where B has full column rank and C is Hermitian and positive-definite

More information

Linear Algebra: Matrix Eigenvalue Problems

Linear Algebra: Matrix Eigenvalue Problems CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given

More information

Lecture IX. Definition 1 A non-singular Sturm 1 -Liouville 2 problem consists of a second order linear differential equation of the form.

Lecture IX. Definition 1 A non-singular Sturm 1 -Liouville 2 problem consists of a second order linear differential equation of the form. Lecture IX Abstract When solving PDEs it is often necessary to represent the solution in terms of a series of orthogonal functions. One way to obtain an orthogonal family of functions is by solving a particular

More information

The University of Texas at Austin Department of Electrical and Computer Engineering. EE381V: Large Scale Learning Spring 2013.

The University of Texas at Austin Department of Electrical and Computer Engineering. EE381V: Large Scale Learning Spring 2013. The University of Texas at Austin Department of Electrical and Computer Engineering EE381V: Large Scale Learning Spring 2013 Assignment Two Caramanis/Sanghavi Due: Tuesday, Feb. 19, 2013. Computational

More information

Fundamentals of Matrices

Fundamentals of Matrices Maschinelles Lernen II Fundamentals of Matrices Christoph Sawade/Niels Landwehr/Blaine Nelson Tobias Scheffer Matrix Examples Recap: Data Linear Model: f i x = w i T x Let X = x x n be the data matrix

More information

(Linear equations) Applied Linear Algebra in Geoscience Using MATLAB

(Linear equations) Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB (Linear equations) Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots

More information

Lecture 15, 16: Diagonalization

Lecture 15, 16: Diagonalization Lecture 15, 16: Diagonalization Motivation: Eigenvalues and Eigenvectors are easy to compute for diagonal matrices. Hence, we would like (if possible) to convert matrix A into a diagonal matrix. Suppose

More information

. = V c = V [x]v (5.1) c 1. c k

. = V c = V [x]v (5.1) c 1. c k Chapter 5 Linear Algebra It can be argued that all of linear algebra can be understood using the four fundamental subspaces associated with a matrix Because they form the foundation on which we later work,

More information

Summary of Week 9 B = then A A =

Summary of Week 9 B = then A A = Summary of Week 9 Finding the square root of a positive operator Last time we saw that positive operators have a unique positive square root We now briefly look at how one would go about calculating the

More information

Linear Algebra- Final Exam Review

Linear Algebra- Final Exam Review Linear Algebra- Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.

More information

Strauss PDEs 2e: Section Exercise 4 Page 1 of 6

Strauss PDEs 2e: Section Exercise 4 Page 1 of 6 Strauss PDEs 2e: Section 5.3 - Exercise 4 Page of 6 Exercise 4 Consider the problem u t = ku xx for < x < l, with the boundary conditions u(, t) = U, u x (l, t) =, and the initial condition u(x, ) =, where

More information

Exercise Sheet 1. 1 Probability revision 1: Student-t as an infinite mixture of Gaussians

Exercise Sheet 1. 1 Probability revision 1: Student-t as an infinite mixture of Gaussians Exercise Sheet 1 1 Probability revision 1: Student-t as an infinite mixture of Gaussians Show that an infinite mixture of Gaussian distributions, with Gamma distributions as mixing weights in the following

More information

The Singular Value Decomposition

The Singular Value Decomposition The Singular Value Decomposition An Important topic in NLA Radu Tiberiu Trîmbiţaş Babeş-Bolyai University February 23, 2009 Radu Tiberiu Trîmbiţaş ( Babeş-Bolyai University)The Singular Value Decomposition

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

Linear Algebra using Dirac Notation: Pt. 2

Linear Algebra using Dirac Notation: Pt. 2 Linear Algebra using Dirac Notation: Pt. 2 PHYS 476Q - Southern Illinois University February 6, 2018 PHYS 476Q - Southern Illinois University Linear Algebra using Dirac Notation: Pt. 2 February 6, 2018

More information

Finite Difference Methods for Boundary Value Problems

Finite Difference Methods for Boundary Value Problems Finite Difference Methods for Boundary Value Problems October 2, 2013 () Finite Differences October 2, 2013 1 / 52 Goals Learn steps to approximate BVPs using the Finite Difference Method Start with two-point

More information

Lecture 7: Positive Semidefinite Matrices

Lecture 7: Positive Semidefinite Matrices Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.

More information

Dimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas

Dimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

4. Linear transformations as a vector space 17

4. Linear transformations as a vector space 17 4 Linear transformations as a vector space 17 d) 1 2 0 0 1 2 0 0 1 0 0 0 1 2 3 4 32 Let a linear transformation in R 2 be the reflection in the line = x 2 Find its matrix 33 For each linear transformation

More information

AMS10 HW7 Solutions. All credit is given for effort. (-5 pts for any missing sections) Problem 1 (20 pts) Consider the following matrix 2 A =

AMS10 HW7 Solutions. All credit is given for effort. (-5 pts for any missing sections) Problem 1 (20 pts) Consider the following matrix 2 A = AMS1 HW Solutions All credit is given for effort. (- pts for any missing sections) Problem 1 ( pts) Consider the following matrix 1 1 9 a. Calculate the eigenvalues of A. Eigenvalues are 1 1.1, 9.81,.1

More information

Flow Instability Matlab Tutorial Bangalore, January 2010 Matthew Juniper, Dan Henningson, Peter Schmid 1

Flow Instability Matlab Tutorial Bangalore, January 2010 Matthew Juniper, Dan Henningson, Peter Schmid 1 Flow Instability Matlab Tutorial Bangalore, January Matthew Juniper, Dan Henningson, Peter Schmid Simple differentiation matrices Imagine a vector that represents the values of a function y at N equispaced

More information

Ch 10.1: Two Point Boundary Value Problems

Ch 10.1: Two Point Boundary Value Problems Ch 10.1: Two Point Boundary Value Problems In many important physical problems there are two or more independent variables, so the corresponding mathematical models involve partial differential equations.

More information

Homework Set 4 - SOLUTIONS

Homework Set 4 - SOLUTIONS MATH 55 Scientific Computing II Spring 6 Homework Set 4 - SOLUTIONS. For N even define a discrete grid x h for [, π] by x k = k h where h = π/n and k N. Show e i( N +j)x k = e i( N j)x k for j =,,...,

More information

Systems of Algebraic Equations and Systems of Differential Equations

Systems of Algebraic Equations and Systems of Differential Equations Systems of Algebraic Equations and Systems of Differential Equations Topics: 2 by 2 systems of linear equations Matrix expression; Ax = b Solving 2 by 2 homogeneous systems Functions defined on matrices

More information

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the

More information

Singular Value Decomposition

Singular Value Decomposition Singular Value Decomposition CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Singular Value Decomposition 1 / 35 Understanding

More information

Assignment #10: Diagonalization of Symmetric Matrices, Quadratic Forms, Optimization, Singular Value Decomposition. Name:

Assignment #10: Diagonalization of Symmetric Matrices, Quadratic Forms, Optimization, Singular Value Decomposition. Name: Assignment #10: Diagonalization of Symmetric Matrices, Quadratic Forms, Optimization, Singular Value Decomposition Due date: Friday, May 4, 2018 (1:35pm) Name: Section Number Assignment #10: Diagonalization

More information

Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Dr Gerhard Roth COMP 40A Winter 05 Version Linear algebra Is an important area of mathematics It is the basis of computer vision Is very widely taught, and there are many resources

More information

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1

More information

Basic Calculus Review

Basic Calculus Review Basic Calculus Review Lorenzo Rosasco ISML Mod. 2 - Machine Learning Vector Spaces Functionals and Operators (Matrices) Vector Space A vector space is a set V with binary operations +: V V V and : R V

More information

STA141C: Big Data & High Performance Statistical Computing

STA141C: Big Data & High Performance Statistical Computing STA141C: Big Data & High Performance Statistical Computing Numerical Linear Algebra Background Cho-Jui Hsieh UC Davis May 15, 2018 Linear Algebra Background Vectors A vector has a direction and a magnitude

More information

Cheat Sheet for MATH461

Cheat Sheet for MATH461 Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A

More information

Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 1 / 14 Introduction We define eigenvalues and eigenvectors. We discuss how to

More information

Econ Slides from Lecture 7

Econ Slides from Lecture 7 Econ 205 Sobel Econ 205 - Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for

More information

Lecture notes: Applied linear algebra Part 1. Version 2

Lecture notes: Applied linear algebra Part 1. Version 2 Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and

More information

ANSWERS. E k E 2 E 1 A = B

ANSWERS. E k E 2 E 1 A = B MATH 7- Final Exam Spring ANSWERS Essay Questions points Define an Elementary Matrix Display the fundamental matrix multiply equation which summarizes a sequence of swap, combination and multiply operations,

More information

Stat 159/259: Linear Algebra Notes

Stat 159/259: Linear Algebra Notes Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the

More information

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015 Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

More information

Linear Algebra & Geometry why is linear algebra useful in computer vision?

Linear Algebra & Geometry why is linear algebra useful in computer vision? Linear Algebra & Geometry why is linear algebra useful in computer vision? References: -Any book on linear algebra! -[HZ] chapters 2, 4 Some of the slides in this lecture are courtesy to Prof. Octavia

More information

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation

More information

Recall : Eigenvalues and Eigenvectors

Recall : Eigenvalues and Eigenvectors Recall : Eigenvalues and Eigenvectors Let A be an n n matrix. If a nonzero vector x in R n satisfies Ax λx for a scalar λ, then : The scalar λ is called an eigenvalue of A. The vector x is called an eigenvector

More information

Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58

Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58 Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms

More information

Linear Algebra - Part II

Linear Algebra - Part II Linear Algebra - Part II Projection, Eigendecomposition, SVD (Adapted from Sargur Srihari s slides) Brief Review from Part 1 Symmetric Matrix: A = A T Orthogonal Matrix: A T A = AA T = I and A 1 = A T

More information

Find the solution set of 2x 3y = 5. Answer: We solve for x = (5 + 3y)/2. Hence the solution space consists of all vectors of the form

Find the solution set of 2x 3y = 5. Answer: We solve for x = (5 + 3y)/2. Hence the solution space consists of all vectors of the form Math 2 Homework #7 March 4, 2 7.3.3. Find the solution set of 2x 3y = 5. Answer: We solve for x = (5 + 3y/2. Hence the solution space consists of all vectors of the form ( ( ( ( x (5 + 3y/2 5/2 3/2 x =

More information

The following definition is fundamental.

The following definition is fundamental. 1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic

More information

Practice Final Exam Solutions for Calculus II, Math 1502, December 5, 2013

Practice Final Exam Solutions for Calculus II, Math 1502, December 5, 2013 Practice Final Exam Solutions for Calculus II, Math 5, December 5, 3 Name: Section: Name of TA: This test is to be taken without calculators and notes of any sorts. The allowed time is hours and 5 minutes.

More information

Linear Algebra Primer

Linear Algebra Primer Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

More information

MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, Chapter 1: Linear Equations and Matrices MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

More information

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0. Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the

More information

Ordinary Differential Equations II

Ordinary Differential Equations II Ordinary Differential Equations II February 23 2017 Separation of variables Wave eq. (PDE) 2 u t (t, x) = 2 u 2 c2 (t, x), x2 c > 0 constant. Describes small vibrations in a homogeneous string. u(t, x)

More information

ANSWERS (5 points) Let A be a 2 2 matrix such that A =. Compute A. 2

ANSWERS (5 points) Let A be a 2 2 matrix such that A =. Compute A. 2 MATH 7- Final Exam Sample Problems Spring 7 ANSWERS ) ) ). 5 points) Let A be a matrix such that A =. Compute A. ) A = A ) = ) = ). 5 points) State ) the definition of norm, ) the Cauchy-Schwartz inequality

More information

c c c c c c c c c c a 3x3 matrix C= has a determinant determined by

c c c c c c c c c c a 3x3 matrix C= has a determinant determined by Linear Algebra Determinants and Eigenvalues Introduction: Many important geometric and algebraic properties of square matrices are associated with a single real number revealed by what s known as the determinant.

More information

Chap 3. Linear Algebra

Chap 3. Linear Algebra Chap 3. Linear Algebra Outlines 1. Introduction 2. Basis, Representation, and Orthonormalization 3. Linear Algebraic Equations 4. Similarity Transformation 5. Diagonal Form and Jordan Form 6. Functions

More information

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017 Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...

More information

WI1403-LR Linear Algebra. Delft University of Technology

WI1403-LR Linear Algebra. Delft University of Technology WI1403-LR Linear Algebra Delft University of Technology Year 2013 2014 Michele Facchinelli Version 10 Last modified on February 1, 2017 Preface This summary was written for the course WI1403-LR Linear

More information

1 Matrices and vector spaces

1 Matrices and vector spaces Matrices and vector spaces. Which of the following statements about linear vector spaces are true? Where a statement is false, give a counter-example to demonstrate this. (a) Non-singular N N matrices

More information

Lecture Notes for Math 251: ODE and PDE. Lecture 30: 10.1 Two-Point Boundary Value Problems

Lecture Notes for Math 251: ODE and PDE. Lecture 30: 10.1 Two-Point Boundary Value Problems Lecture Notes for Math 251: ODE and PDE. Lecture 30: 10.1 Two-Point Boundary Value Problems Shawn D. Ryan Spring 2012 Last Time: We finished Chapter 9: Nonlinear Differential Equations and Stability. Now

More information

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

More information

Lecture notes on Quantum Computing. Chapter 1 Mathematical Background

Lecture notes on Quantum Computing. Chapter 1 Mathematical Background Lecture notes on Quantum Computing Chapter 1 Mathematical Background Vector states of a quantum system with n physical states are represented by unique vectors in C n, the set of n 1 column vectors 1 For

More information

33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM

33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM 33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM (UPDATED MARCH 17, 2018) The final exam will be cumulative, with a bit more weight on more recent material. This outline covers the what we ve done since the

More information