# Recall the convention that, for us, all vectors are column vectors.

Save this PDF as:

Size: px
Start display at page:

Download "Recall the convention that, for us, all vectors are column vectors."

## Transcription

1 Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists a real and nonzero vector called an eigenvector for λ such that A = λ. Whenever is an eigenvector for λ, so is for every real number. The characteristic polynomial χ A of matrix A is the function χ A (λ) := det(λi A) defined for all complex numbers λ, where I denotes the identity matrix. It is not hard to see that a complex number λ is an eigenvalue of A if and only if χ A (λ) =0. We see by direct computation that χ A is an th-order polynomial. Therefore, A has precisely eigenvalues, thanks to the fundamental theorem of algebra. We can write them as λ 1 λ, or sometimes more precisely as λ 1 (A)λ (A). 1. The spectral theorem. The following important theorem is the starting point of our discussion. It might help to recall that vectors 1 R are orthonormal if =0when = and = 2 =1. Theorem 1. If A is a real and symmetric matrix, then λ 1 λ are real numbers. Moreover, there exist orthonormal eigenvectors 1 that correspond respectively to λ 1 λ. 11

2 12 3. Some linear algebra I will not prove this result, as it requires developing a good deal of elementary linear algebra that we will not need. Instead, let me state and prove a result that is central for us. Theorem 2 (The spectral theorem). Let A denote a symmetric matrix with real eigenvalues λ 1 λ and corresponding orthonormal eigenvectors 1. Define D := diag(λ 1 λ ) to be the diagonal matrix of the λ s and P to be the matrix whose columns are 1 though respectively; that is, λ λ D := 0 0 λ 3 0 P := ( 1 ) λ Then P is orthogonal [P = P 1 ] and A = PDP 1 = PDP. Proof. P is orthogonal because the orthonormality of the s implies that P P = 1. ( 1 ) = I Furthermore, because A = λ, it follows that AP = PD, which is another way to say that A = PDP 1. Recall that the trace of an matrix A is the sum A A of its diagonal entries. Corollary 3. If A is a real and symmetric matrix with real eigenvalues λ 1 λ, then tr(a) =λ λ and det(a) =λ 1 λ Proof. Write A, in spectral form, as PDP 1. Since the determinant of P 1 is the reciprocal of that of A, it follows that det(a) = det(d), which is clearly λ 1 λ. In order to compute the trace of A we compute directly also: tr(a) = P DP 1 = P D P 1 =1 =1 =1 =1 =1 =1 =1 = PP 1 D = D = tr(d) =1

3 2. Positive-semidefinite matrices 13 which is λ λ. 2. The square-root matrix. Let A continue to denote a real and symmetric matrix. Proposition 4. There exists a complex and symmetric matrix B called the square root of A and written as A 1 /2 or even sometimes as A such that A = B 2 := BB. The proof of Proposition 4 is more important than its statement. So let us prove this result. Proof. Apply the spectral theorem and write A = PDP 1. Since D is a diagonal matrix, its square root can be defined unambiguously as the following complex-valued diagonal matrix: λ 1 / λ 1 / D 1 /2 := 0 0 λ 1 / λ 1 /2 Define B := PD 1 /2 P 1, and note that since P 1 P = I and (D 1 /2 ) 2 = D. B 2 = PD 1 /2 P 1 PD 1 /2 P 1 = PDP 1 = A 2. Positive-semidefinite matrices Recall that an matrix A is positive semidefinite if it is symmetric and A 0 for all R Recall that A is positive definite if it is symmetric and A > 0 for all nonzero R Theorem 5. A symmetric matrix A is positive semidefinite if and only if all of its eigenvalues are 0. A is positive definite if and only if all of its eigenvalues are > 0. In the latter case, A is also nonsingular. The following is a ready consequence. Corollary 6. All of the eigenvalues of a variance-covariance matrix are always 0. Now let us establish the theorem.

4 14 3. Some linear algebra Proof of Theorem 5. Suppose A is positive semidefinite, and let λ denote one of its eignenvalues, together with corresponding eigenvector. Since 0 A = λ 2 and > 0, it follows that λ 0. This proves that all of the eigenvalues of A are nonnegative. If A is positive definite, then the same argument shows that all of its eigenvalues are > 0. Because det(a) is the product of all eigenvalues of A (Corollary 3), it follows that det(a) > 0, whence A is nonsingular. This proves slightly more than half of the proposition. Now let us suppose that all eigenvalues of A are 0. We write A in spectral form A = PDP, and observe that D is a diagonal matrix of nonnegative numbers. By virute of its construction. A 1 /2 = PD 1 /2 P, and hence for all R, A = D 1 /2 P D 1 /2 P = D 1 /2 P which is 0. Therefore, A is positive semidefinite. 2 (1) If all of the eigenvalues of A are > 0, then (1) tells us that A = D 1 /2 P 2 2 = D 1 /2 2 P = λ [P] (2) =1 where λ > 0 for all. Therefore, A min 1 λ =1 =1 [P] 2 = min 1 λ P P = min 1 λ 2 Since min 1 λ > 0, it follows that A > 0 for all nonzero. This completes the proof. Let us pause and point out a consequence of the proof of this last result. Corollary 7. If A is positive semidefinite, then its extremal eigenvalues satisfy min λ A = min 1 >0 2 max λ A = max 1 >0 2 Proof. We saw, during the course of the previous proof, that Optimize over all to see that min λ 2 A for all R (3) 1 min λ A min 1 >0 2 (4)

5 3. The rank of a matrix 15 But min 1 λ is an eigenvalue for A; let denote a corresponding eigenvector in order to see that min λ A min 1 >0 2 A 2 = min 1 λ So both inequalities are in fact equalities, and hence follows the formula for the minimum eigenvalue. The one for the maximum eigenvalue is proved similarly. Finally, a word about the square root of positive semidefinite matrices: Proposition 8. If A is positive semidefinite, then so is A 1 /2. positive definite, then so is A 1 /2. If A is Proof. We write, in spectral form, A = PDP and observe [by squaring it] that A 1 /2 = PD 1 /2 P. Note that D 1 /2 is a real diagonal matrix since the eigenvalues of A are 0. Therefore, we may apply (1) to A 1 /2 [in place of A] to see that A 1 /2 = D 1 /4 P 2 0 where D 1 /4 denotes the [real] square root of D 1 /2. This proves that if A is positive semidefinite, then so is A 1 /2. Now suppose there exists a positive definite A whose square root is not positive definite. It would follow that there necessarily exists a nonzero R such that A 1 /2 = D 1 /4 P 2 =0. Since D 1 /4 P = 0, D 1 /2 P = D 1 /4 D 1 /4 P = 0 A = D 1 /2 P 2 =0 And this contradicts the assumption that A is positive definite. 3. The rank of a matrix Recall that vectors 1 are linearly independent if = 0 1 = = =0 For instance, 1 := (1 0) and 2 := (0 1) are linearly independent 2- vectors. The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A. We can interpret these definitions geometrically as follows: First, suppose A is and define (A) denote the linear space of all vectors of the form , where 1 are the column vectors of A and 1 are real numbers. We call (A) the column space of A. We can define the row space (A), of A similarly, or simply define (A) := (A ).

6 16 3. Some linear algebra Lemma 9. For every matrix A, (A) ={A : R } (A) := A : R We can think of an matrix A as a mapping from R into R ; namely, we can think of matrix A also as the function A () := A. In this way we see that (A) is also the range of the function A. Proof. Let us write the columns of A as 1 2 Note that (A) if and only if there exist 1 such that = = A, where := ( 1 ). This shows that (A) is the collection of all vectors of the form A, for R. The second assertion [about (A)] follows from the definition of (A equalling (A ) and the alreadyproven first assertion. It then follows, from the definition of dimension, that column rank of A = dim (A) row rank of A = dim (A) Proposition 10. Given any matrix A, its row rank and column rank are the same. We write their common value as rank(a). Proof. Suppose A is and its column rank is. Let 1 denote a basis for (A) and consider the matrix matrix B := ( 1 ). Write A, columnwise, as A := ( 1 ). For every 1, there exists 1 such that = Let C := ( ) be the resulting matrix, and note that A = BC. Because A = =1 B C ;, every row of A is a linear combination of the rows of C. In other words, (A) (C) and hence the row rank of A is dim (C) = = the column rank of A. Apply this fact to A to see that also the row rank of A is the column rank of A ; equivalently that the column rank of A is the row rank of A. Proposition 11. If A is and B is, then rank(ab) min (rank(a) rank(b)) Proof. The proof uses an idea that we exploited already in the proof of Proposition 10: Since (AB) = ν=1 A νb ν, the rows of AB are linear combinations of the rows of B; that is (AB) (B), whence rank(ab) rank(b). Also, (AB) (A), whence rank(ab) rank(a). These observations complete the proof.

7 3. The rank of a matrix 17 Proposition 12. If A and C are nonsingular, then rank(abc) =rank(b) provided that the dimensions match up so that ABC makes sense. Proof. Let D := ABC; our goal is to show that rank(d) =rank(b). Two applications of the previous proposition together yield rank(d) rank(ab) rank(b) And since B = A 1 DC 1, we have also rank(b) rank(a 1 D) rank(d) Corollary 13. If A is an real and symmetric matrix, then rank(a) = the total number of nonzero eigenvalues of A. In particular, A has full rank if and only if A is nonsingular. Finally, (A) is the linear space spanned by the eigenvectors of A that correspond to nonzero eigenvalues. Proof. We write A, in spectral form, as A = PDP 1, and apply the preceding proposition to see that rank(a) =rank(d), which is clearly the total number of nonzero eigenvalue of A. Since A is nonsingular if and only if all of its eigenvalues are nonzero, A has full rank if and only if A is nonsingular. Finally, suppose A has rank ; this is the number of its nonzero eigenvalues λ 1 λ. Let 1 denote orthonormal eigenvectors such that 1 are eigenvectors that correspond to λ 1 λ and +1 are eigenvectors that correspond to eigenvalues 0 [Gram Schmidt]. And define to be the span of 1 ; i.e., := { : 1 R} Our final goal is to prove that = (A), which we know is equal to the linear space of all vectors of the form A. Clearly, = A, where = =1 ( /λ ) Therefore, (A). If =, then this suffices because in that case 1 is a basis for R, hence = (A) =R. If <, then we can write every R as , so that Ax = =1 λ. Thus, (A) and we are done. Let A be and define the null space [or kernel ] of A as (A) := { R : A = 0} Note that (A) is the linear span of the eigenvectors of A that correspond to eigenvalue 0. The other eigenvectors can be chosen to be orthogonal to these, and hence the preceding proof contains the facts

8 18 3. Some linear algebra that: (i) Nonzero elements of (A) are orthogonal to nonzero elements of (A); and (ii) dim (A)+ rank(a) = (= the number of columns of A) (5) Proposition 14. rank(a) = rank(a A) = rank(aa ) for every matrix A. Proof. If A = 0 then A A = 0, and if A A = 0, then A 2 = A A =0 In other words, (A) =(A A). Because A A and A both have columnes, it follows from (5) that rank(a A)=rank(A). Apply this observation to A to see that rank(a )=rank(aa ) as well. The result follows from this and the fact that A and A have the same rank (Proposition 10). 4. Projection matrices A matrix A is said to be a projection matrix if: (i) A is symmetric; and (ii) A is idempotent ; that is, A 2 = A. In- Note that projection matrices are always positive semidefinite. deed, A = A 2 = A A = A 2 0 Proposition 15. If A is an projection matrix, then so is I A. Moreover, all eigenvalues of A are zeros and ones, and rank(a) =the number of eigenvalues that are equal to one. Proof. (I A) 2 = I 2A + A 2 = I A Since I A is symmetric also, it is a projection. If λ is an eigenvalue of A and is a corresponding eigenvector, then λ = A = A 2 = λa = λ 2. Multiply both sides by to see that λ 2 = λ 2 2. Since > 0, it follows that λ {01}. The total number of nonzero eigenvalues is then the total number of eigenvalues that are ones. Therefore, the rank of A is the total number of eigenvalues that are one. Corollary 16. If A is a projection matrix, then rank(a) =tr(a). Proof. Simply recall that tr(a) is the sum of the eigenvalues, which for a projection matrix, is the total number of eigenavalues that are one. Why are they called projection matrices? Or, perhaps even more importantly, what is a projection?

9 4. Projection matrices 19 Lemma 17. Let Ω denote a linear subspace of R, and R be fixed. Then there exists a unique element Ω that is closest to ; that is, = min Ω The point is called the projection of onto Ω. Proof. Let := dim Ω, so that there exists an orthonormal basis 1 for Ω. Extend this to a basis 1 for all of R by the Gram Schmidt method. Given a fixed vector R, we can write it as := for some 1 R. Define := Clearly, Ω and 2 = =+1 2. Any other Ω can be written as = =1, and hence 2 = =1 ( ) 2 + =+1 2, which is strictly greater than 2 = =+1 2 unless = for all =1; i.e., unless =. Usually, we have a -dimensional linear subspace Ω of R that is the range of some matrix A. That is, Ω={A : R }. Equivalently, Ω=(A). In that case, min Ω 2 = min A 2 = min A A A A + R R Because A is a scalar, the preceding is simplified to min Ω 2 = min A A 2 A + R Suppose that the positive semidefinite matrix A A is nonsingular [so that A A and hence also (AA ) 1 are both positive definite]. Then, we can relabel variables [α := A A] to see that min Ω 2 = min α (A A) 1 α 2α (A A) 1 A + α R A little arithmetic shows that (α A ) (A A) 1 (α A ) Consequently, = α (A A) 1 α 2α (A A) 1 A + A(A A) 1 A min 2 Ω = min (α A ) (A A) 1 (α A ) A(A A) 1 A + α R The first term in the parentheses is 0; in fact it is > 0 unless we select α = A. This proves that the projection of onto Ω is obtained by setting α := A, in which case the projection itself is A =

10 20 3. Some linear algebra A(A A) 1 A and the distance between and is the square root of 2 A(A A) 1 A. Let P Ω := A(A A) 1 A. It is easy to see that P Ω is a projection matrix. The preceding shows that P Ω is the projection of onto Ω for every R. That is, we can think of P Ω as the matrix that projects onto Ω. Moreover, the distance between and the linear subspace Ω [i.e., min R ] is exactly the square root of P Ω = (I P Ω ) = (I P Ω ) 2, because I P Ω is a projection matrix. What space does it project into? Let Ω denote the collection of all -vectors that are perpendicular to every element of Ω. If Ω, then we can write, for all R, 2 = (I P Ω ) + P Ω 2 = (I P Ω ) 2 + P Ω 2 2 { (I P Ω )} P Ω = (I P Ω ) 2 + P Ω 2 since is orthogonal to every element of Ω including P Ω, and P Ω = P 2 Ω. Take the minimum over all Ω to find that I P Ω is the projection onto Ω. Let us summarize our findings. Proposition 18. If A A is nonsingular [equivalently, has full rank], then P (A) := A(A A) 1 A is the projection onto (A), I P (A) = P (A) is the projection onto Ω, and we have = P (A) + P (A) and 2 = P (A) 2 + P (A) 2 The last result is called the Pythagorean property.

### MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

### MA 265 FINAL EXAM Fall 2012

MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators

### Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,

2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for

### (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### Elementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th.

Elementary Linear Algebra Review for Exam Exam is Monday, November 6th. The exam will cover sections:.4,..4, 5. 5., 7., the class notes on Markov Models. You must be able to do each of the following. Section.4

### Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

### Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam

Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system

### ft-uiowa-math2550 Assignment OptionalFinalExamReviewMultChoiceMEDIUMlengthForm due 12/31/2014 at 10:36pm CST

me me ft-uiowa-math255 Assignment OptionalFinalExamReviewMultChoiceMEDIUMlengthForm due 2/3/2 at :3pm CST. ( pt) Library/TCNJ/TCNJ LinearSystems/problem3.pg Give a geometric description of the following

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

### 22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices

m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix

### Chapter Two Elements of Linear Algebra

Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to

### (a) II and III (b) I (c) I and III (d) I and II and III (e) None are true.

1 Which of the following statements is always true? I The null space of an m n matrix is a subspace of R m II If the set B = {v 1,, v n } spans a vector space V and dimv = n, then B is a basis for V III

### HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)

HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe

### Solutions to Final Practice Problems Written by Victoria Kala Last updated 12/5/2015

Solutions to Final Practice Problems Written by Victoria Kala vtkala@math.ucsb.edu Last updated /5/05 Answers This page contains answers only. See the following pages for detailed solutions. (. (a x. See

### Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

### Linear Algebra Massoud Malek

CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

### Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007

Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007 You have 1 hour and 20 minutes. No notes, books, or other references. You are permitted to use Maple during this exam, but you must start with a blank

### Preliminary Linear Algebra 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 100

Preliminary Linear Algebra 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 100 Notation for all there exists such that therefore because end of proof (QED) Copyright c 2012

### Chapter 4 & 5: Vector Spaces & Linear Transformations

Chapter 4 & 5: Vector Spaces & Linear Transformations Philip Gressman University of Pennsylvania Philip Gressman Math 240 002 2014C: Chapters 4 & 5 1 / 40 Objective The purpose of Chapter 4 is to think

### and let s calculate the image of some vectors under the transformation T.

Chapter 5 Eigenvalues and Eigenvectors 5. Eigenvalues and Eigenvectors Let T : R n R n be a linear transformation. Then T can be represented by a matrix (the standard matrix), and we can write T ( v) =

### Linear Algebra Practice Problems

Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,

### I. Multiple Choice Questions (Answer any eight)

Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam - Course Instructor : Prashanth L.A. Date : Sep-24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY

### MATH 369 Linear Algebra

Assignment # Problem # A father and his two sons are together 00 years old. The father is twice as old as his older son and 30 years older than his younger son. How old is each person? Problem # 2 Determine

### Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

### 1. Select the unique answer (choice) for each problem. Write only the answer.

MATH 5 Practice Problem Set Spring 7. Select the unique answer (choice) for each problem. Write only the answer. () Determine all the values of a for which the system has infinitely many solutions: x +

### Eigenvalues and Eigenvectors

CHAPTER Eigenvalues and Eigenvectors CHAPTER CONTENTS. Eigenvalues and Eigenvectors 9. Diagonalization. Complex Vector Spaces.4 Differential Equations 6. Dynamical Systems and Markov Chains INTRODUCTION

### Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

### DS-GA 1002 Lecture notes 10 November 23, Linear models

DS-GA 2 Lecture notes November 23, 2 Linear functions Linear models A linear model encodes the assumption that two quantities are linearly related. Mathematically, this is characterized using linear functions.

### Math 314/ Exam 2 Blue Exam Solutions December 4, 2008 Instructor: Dr. S. Cooper. Name:

Math 34/84 - Exam Blue Exam Solutions December 4, 8 Instructor: Dr. S. Cooper Name: Read each question carefully. Be sure to show all of your work and not just your final conclusion. You may not use your

### There are two things that are particularly nice about the first basis

Orthogonality and the Gram-Schmidt Process In Chapter 4, we spent a great deal of time studying the problem of finding a basis for a vector space We know that a basis for a vector space can potentially

### . The following is a 3 3 orthogonal matrix: 2/3 1/3 2/3 2/3 2/3 1/3 1/3 2/3 2/3

Lecture Notes: Orthogonal and Symmetric Matrices Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Orthogonal Matrix Definition. An n n matrix

### EIGENVALUES AND EIGENVECTORS

EIGENVALUES AND EIGENVECTORS Diagonalizable linear transformations and matrices Recall, a matrix, D, is diagonal if it is square and the only non-zero entries are on the diagonal This is equivalent to

### AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES

AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES JOEL A. TROPP Abstract. We present an elementary proof that the spectral radius of a matrix A may be obtained using the formula ρ(a) lim

### ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA

ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA Kent State University Department of Mathematical Sciences Compiled and Maintained by Donald L. White Version: August 29, 2017 CONTENTS LINEAR ALGEBRA AND

### 5.) For each of the given sets of vectors, determine whether or not the set spans R 3. Give reasons for your answers.

Linear Algebra - Test File - Spring Test # For problems - consider the following system of equations. x + y - z = x + y + 4z = x + y + 6z =.) Solve the system without using your calculator..) Find the

### MATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2.

MATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2. Diagonalization Let L be a linear operator on a finite-dimensional vector space V. Then the following conditions are equivalent:

### Chapter 3. Matrices. 3.1 Matrices

40 Chapter 3 Matrices 3.1 Matrices Definition 3.1 Matrix) A matrix A is a rectangular array of m n real numbers {a ij } written as a 11 a 12 a 1n a 21 a 22 a 2n A =.... a m1 a m2 a mn The array has m rows

### Math Linear Algebra Final Exam Review Sheet

Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

### 4. Linear transformations as a vector space 17

4 Linear transformations as a vector space 17 d) 1 2 0 0 1 2 0 0 1 0 0 0 1 2 3 4 32 Let a linear transformation in R 2 be the reflection in the line = x 2 Find its matrix 33 For each linear transformation

### BASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x

BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,

### Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors week -2 Fall 26 Eigenvalues and eigenvectors The most simple linear transformation from R n to R n may be the transformation of the form: T (x,,, x n ) (λ x, λ 2,, λ n x n

### Computational math: Assignment 1

Computational math: Assignment 1 Thanks Ting Gao for her Latex file 11 Let B be a 4 4 matrix to which we apply the following operations: 1double column 1, halve row 3, 3add row 3 to row 1, 4interchange

### Linear Algebra: Sample Questions for Exam 2

Linear Algebra: Sample Questions for Exam 2 Instructions: This is not a comprehensive review: there are concepts you need to know that are not included. Be sure you study all the sections of the book and

### Span and Linear Independence

Span and Linear Independence It is common to confuse span and linear independence, because although they are different concepts, they are related. To see their relationship, let s revisit the previous

### NONCOMMUTATIVE POLYNOMIAL EQUATIONS. Edward S. Letzter. Introduction

NONCOMMUTATIVE POLYNOMIAL EQUATIONS Edward S Letzter Introduction My aim in these notes is twofold: First, to briefly review some linear algebra Second, to provide you with some new tools and techniques

### Symmetric matrices and dot products

Symmetric matrices and dot products Proposition An n n matrix A is symmetric iff, for all x, y in R n, (Ax) y = x (Ay). Proof. If A is symmetric, then (Ax) y = x T A T y = x T Ay = x (Ay). If equality

### Stat 159/259: Linear Algebra Notes

Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the

### av 1 x 2 + 4y 2 + xy + 4z 2 = 16.

74 85 Eigenanalysis The subject of eigenanalysis seeks to find a coordinate system, in which the solution to an applied problem has a simple expression Therefore, eigenanalysis might be called the method

### Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =

30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can

### Math 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.

Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses

### Optimization Theory. A Concise Introduction. Jiongmin Yong

October 11, 017 16:5 ws-book9x6 Book Title Optimization Theory 017-08-Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 017-08-Lecture Notes page Optimization

### Linear Algebra Problems

Math 504 505 Linear Algebra Problems Jerry L. Kazdan Topics 1 Basics 2 Linear Equations 3 Linear Maps 4 Rank One Matrices 5 Algebra of Matrices 6 Eigenvalues and Eigenvectors 7 Inner Products and Quadratic

### Lecture 3: Review of Linear Algebra

ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,

### Math 225 Linear Algebra II Lecture Notes. John C. Bowman University of Alberta Edmonton, Canada

Math 225 Linear Algebra II Lecture Notes John C Bowman University of Alberta Edmonton, Canada March 23, 2017 c 2010 John C Bowman ALL RIGHTS RESERVED Reproduction of these lecture notes in any form, in

### EE263: Introduction to Linear Dynamical Systems Review Session 5

EE263: Introduction to Linear Dynamical Systems Review Session 5 Outline eigenvalues and eigenvectors diagonalization matrix exponential EE263 RS5 1 Eigenvalues and eigenvectors we say that λ C is an eigenvalue

### Eigenvalues, Eigenvectors, and Diagonalization

Math 240 TA: Shuyi Weng Winter 207 February 23, 207 Eigenvalues, Eigenvectors, and Diagonalization The concepts of eigenvalues, eigenvectors, and diagonalization are best studied with examples. We will

### Topic 2 Quiz 2. choice C implies B and B implies C. correct-choice C implies B, but B does not imply C

Topic 1 Quiz 1 text A reduced row-echelon form of a 3 by 4 matrix can have how many leading one s? choice must have 3 choice may have 1, 2, or 3 correct-choice may have 0, 1, 2, or 3 choice may have 0,

### Linear Algebra Primer

Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

### Math 240 Calculus III

The Calculus III Summer 2015, Session II Wednesday, July 8, 2015 Agenda 1. of the determinant 2. determinants 3. of determinants What is the determinant? Yesterday: Ax = b has a unique solution when A

### Exam in TMA4110 Calculus 3, June 2013 Solution

Norwegian University of Science and Technology Department of Mathematical Sciences Page of 8 Exam in TMA4 Calculus 3, June 3 Solution Problem Let T : R 3 R 3 be a linear transformation such that T = 4,

### c c c c c c c c c c a 3x3 matrix C= has a determinant determined by

Linear Algebra Determinants and Eigenvalues Introduction: Many important geometric and algebraic properties of square matrices are associated with a single real number revealed by what s known as the determinant.

### THE MINIMAL POLYNOMIAL AND SOME APPLICATIONS

THE MINIMAL POLYNOMIAL AND SOME APPLICATIONS KEITH CONRAD. Introduction The easiest matrices to compute with are the diagonal ones. The sum and product of diagonal matrices can be computed componentwise

### Math 215 HW #9 Solutions

Math 5 HW #9 Solutions. Problem 4.4.. If A is a 5 by 5 matrix with all a ij, then det A. Volumes or the big formula or pivots should give some upper bound on the determinant. Answer: Let v i be the ith

### 2. Matrix Algebra and Random Vectors

2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns

### Linear Algebra. Workbook

Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx

### Symmetric Matrices and Eigendecomposition

Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions

### Linear algebra II Homework #1 solutions A = This means that every eigenvector with eigenvalue λ = 1 must have the form

Linear algebra II Homework # solutions. Find the eigenvalues and the eigenvectors of the matrix 4 6 A =. 5 Since tra = 9 and deta = = 8, the characteristic polynomial is f(λ) = λ (tra)λ+deta = λ 9λ+8 =

### 18.06 Problem Set 8 - Solutions Due Wednesday, 14 November 2007 at 4 pm in

806 Problem Set 8 - Solutions Due Wednesday, 4 November 2007 at 4 pm in 2-06 08 03 Problem : 205+5+5+5 Consider the matrix A 02 07 a Check that A is a positive Markov matrix, and find its steady state

### Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v

### 1 Invariant subspaces

MATH 2040 Linear Algebra II Lecture Notes by Martin Li Lecture 8 Eigenvalues, eigenvectors and invariant subspaces 1 In previous lectures we have studied linear maps T : V W from a vector space V to another

### Eigenvectors and Hermitian Operators

7 71 Eigenvalues and Eigenvectors Basic Definitions Let L be a linear operator on some given vector space V A scalar λ and a nonzero vector v are referred to, respectively, as an eigenvalue and corresponding

### Linear Algebra in Actuarial Science: Slides to the lecture

Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations

### 18.06SC Final Exam Solutions

18.06SC Final Exam Solutions 1 (4+7=11 pts.) Suppose A is 3 by 4, and Ax = 0 has exactly 2 special solutions: 1 2 x 1 = 1 and x 2 = 1 1 0 0 1 (a) Remembering that A is 3 by 4, find its row reduced echelon

### Solutions to Math 51 First Exam April 21, 2011

Solutions to Math 5 First Exam April,. ( points) (a) Give the precise definition of a (linear) subspace V of R n. (4 points) A linear subspace V of R n is a subset V R n which satisfies V. If x, y V then

### Section 3.3. Matrix Rank and the Inverse of a Full Rank Matrix

3.3. Matrix Rank and the Inverse of a Full Rank Matrix 1 Section 3.3. Matrix Rank and the Inverse of a Full Rank Matrix Note. The lengthy section (21 pages in the text) gives a thorough study of the rank

### 2. Review of Linear Algebra

2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

### Spectral radius, symmetric and positive matrices

Spectral radius, symmetric and positive matrices Zdeněk Dvořák April 28, 2016 1 Spectral radius Definition 1. The spectral radius of a square matrix A is ρ(a) = max{ λ : λ is an eigenvalue of A}. For an

### 4.3 - Linear Combinations and Independence of Vectors

- Linear Combinations and Independence of Vectors De nitions, Theorems, and Examples De nition 1 A vector v in a vector space V is called a linear combination of the vectors u 1, u,,u k in V if v can be

### Basic Concepts in Matrix Algebra

Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1

### A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

### Linear Algebra Final Exam Study Guide Solutions Fall 2012

. Let A = Given that v = 7 7 67 5 75 78 Linear Algebra Final Exam Study Guide Solutions Fall 5 explain why it is not possible to diagonalize A. is an eigenvector for A and λ = is an eigenvalue for A diagonalize

Fundamentals of Linear Algebra Marcel B. Finan Arkansas Tech University c All Rights Reserved 2 PREFACE Linear algebra has evolved as a branch of mathematics with wide range of applications to the natural

### Gaussian elimination

Gaussian elimination October 14, 2013 Contents 1 Introduction 1 2 Some definitions and examples 2 3 Elementary row operations 7 4 Gaussian elimination 11 5 Rank and row reduction 16 6 Some computational

### Mathematical Methods wk 2: Linear Operators

John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm

### Dimension and Structure

96 Chapter 7 Dimension and Structure 7.1 Basis and Dimensions Bases for Subspaces Definition 7.1.1. A set of vectors in a subspace V of R n is said to be a basis for V if it is linearly independent and

### Singular Value Decomposition

Chapter 6 Singular Value Decomposition In Chapter 5, we derived a number of algorithms for computing the eigenvalues and eigenvectors of matrices A R n n. Having developed this machinery, we complete our

### 4 ORTHOGONALITY ORTHOGONALITY OF THE FOUR SUBSPACES 4.1

4 ORTHOGONALITY ORTHOGONALITY OF THE FOUR SUBSPACES 4.1 Two vectors are orthogonal when their dot product is zero: v w = orv T w =. This chapter moves up a level, from orthogonal vectors to orthogonal

### 33A Linear Algebra and Applications: Practice Final Exam - Solutions

33A Linear Algebra and Applications: Practice Final Eam - Solutions Question Consider a plane V in R 3 with a basis given by v = and v =. Suppose, y are both in V. (a) [3 points] If [ ] B =, find. (b)

### Diagonalisierung. Eigenwerte, Eigenvektoren, Mathematische Methoden der Physik I. Vorlesungsnotizen zu

Eigenwerte, Eigenvektoren, Diagonalisierung Vorlesungsnotizen zu Mathematische Methoden der Physik I J. Mark Heinzle Gravitational Physics, Faculty of Physics University of Vienna Version /6/29 2 version

### The Multivariate Gaussian Distribution

The Multivariate Gaussian Distribution Chuong B. Do October, 8 A vector-valued random variable X = T X X n is said to have a multivariate normal or Gaussian) distribution with mean µ R n and covariance

### SVD and its Application to Generalized Eigenvalue Problems. Thomas Melzer

SVD and its Application to Generalized Eigenvalue Problems Thomas Melzer June 8, 2004 Contents 0.1 Singular Value Decomposition.................. 2 0.1.1 Range and Nullspace................... 3 0.1.2

### Lecture Note 1: Background

ECE5463: Introduction to Robotics Lecture Note 1: Background Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 1 (ECE5463 Sp18)

### 5.3.5 The eigenvalues are 3, 2, 3 (i.e., the diagonal entries of D) with corresponding eigenvalues. Null(A 3I) = Null( ), 0 0

535 The eigenvalues are 3,, 3 (ie, the diagonal entries of D) with corresponding eigenvalues,, 538 The matrix is upper triangular so the eigenvalues are simply the diagonal entries, namely 3, 3 The corresponding

### Abstract Vector Spaces and Concrete Examples

LECTURE 18 Abstract Vector Spaces and Concrete Examples Our discussion of linear algebra so far has been devoted to discussing the relations between systems of linear equations, matrices, and vectors.

### 4. Linear Subspaces Addition and scaling

71 4 Linear Subspaces There are many subsets of R n which mimic R n For example, a plane L passing through the origin in R 3 actually mimics R 2 in many ways First, L contains zero vector O as R 2 does

### Lecture Note 1: Background

ECE5463: Introduction to Robotics Lecture Note 1: Background Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 1 (ECE5463 Sp18)