Christopher Engström November 14, 2014 Hermitian LU QR echelon
Contents of todays lecture Some interesting / useful / important of matrices Hermitian LU QR echelon
Rewriting a as a product of several matrices. Choosing these factor matrices wisely can make problems easier to solve. Also known as decomposition Hermitian LU QR echelon
Diagonalizable Definition If B = S 1 DS where D is a diagonal then B is diagonalizable. Motivation. Using elementary row operations we want to turn Bx = y into Dˆx = ŷ. This can be written as SBx = Sy. Since elementary row operations are invertible SBS 1 Sx = Sy. Let ˆx = Sx and ŷ = Sy, then D = SBS 1 B = S 1 DS Hermitian LU QR echelon
... 0... A =...... 0 0... Hermitian LU QR echelon
Can be lower (left) or upper (right) triangular Easy to solve equation systems involving triangular matrices Diagonal values are also eigenvalues Hermitian LU QR echelon
0 A = 0 0......... 0 0 0 0 0 0 0 Hermitian LU QR echelon
Almost triangular Multiplication of a (upper) matrices and a (upper) triangular gives a new (Useful in for example the QR-method used to find eigenvalues of a ). Diagonal elements usually give a rough approximation of the eigenvalues. Hermitian LU QR echelon
Hermitian Definition The Hermitian conjugate of a A is denoted A H and is defined by (A H ) ij = (A) ji. Definition A is said to be Hermitian (or self-adjoint) if A H = A Hermitian LU QR echelon
Hermitian Notice the similarities with a symmetric A = A. All eigenvalues real. Always diagonalizable. Important in theoretical physics, quantum physics, electroengineering and in certain problems in statistics. Hermitian LU QR echelon
Definition A, A, is said to be unitary if A H = A 1. Hermitian LU QR echelon
Properties of unitary matrices Theorem Let U be a unitary, then a) U is always invertible. b) U 1 is also unitary. c) det(u) = 1 d) (UV) H = (UV) 1 if V is also unitary. e) For any λ that is an eigenvalue of U, λ = e iω, 0 ω 2π. f) Let v be a vector, then Uv = v (for any vector norm). g) The rows/columns of U are orthonormal, that is U i. U H j. = 0, i j, U k. U H k. = 1. h) U preserves eigenvalues. Hermitian LU QR echelon
Example of a unitary The C below rotates a vector by the angle θ around the x-axis 1 0 0 C = 0 cos(θ) sin(θ) 0 sin(θ) cos(θ) and is a unitary. Hermitian LU QR echelon
Definition We consider a square symmetric real valued n n A, then: A is positive definite if x Ax is positive for all non-zero vectors x. A is positive semidefinite if x Ax is non-negative for all non-zero vectors x. A is positive definite λ > 0 for all λ eigenvalue of A. Can also define negative definite and semi-definite matrices. Hermitian LU QR echelon
matrices have many useful, if A is positive definite then A is invertible. A have a unique cholesky decomposition (seen later today). matrices are closely related to quadratic (last lecture). Any Covariance is positive semi-definite. Hermitian LU QR echelon
Diagonalizable A = S 1 DS with D diagonal Other important s: QΛQ 1 LU- GG H QR- CF Jordan canonical S 1 JS UΣV H Hermitian LU QR echelon
is a special version of diagonal. It is sometimes referred to as eigendecomposition. Let A be an square (n n) with linearly independent rows. Then A = QΛQ 1 where AQ.i = Λ ii Q.i for all 1 i n. Hermitian LU QR echelon
Let A be an m n with rank(a) = r (A has r independent rows/columns). Then A = CF where C M m r and F M r n Hermitian LU QR echelon
How can we find this? Rewrite on reduced row echelon 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 B = 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Hermitian LU QR echelon
Create C by removing all columns in A that corresponds to a non-pivot column in B. In this example C = [ A.2 A.4 A.5 A.6 A.8 ] Create F by removing all zero rows in B. In this example F = [ B 1. B 2. B 3. B 4. B 5. ] Hermitian LU QR echelon
LU- A = LR = LU. L is a n n lower triangular. U is a n m upper triangular. Solve Ax = L(Ux) = b by first solving Ly = b and then solve Ux = y. Both these systems are easy to solve since L and U are both triangular. Not every A have a LU, not even every square invertible. Hermitian LU QR echelon
LUP- Theorem Every n m A have a. where PA = LU P is a n n permutation. L is a n n lower triangular. U is a n m upper triangular. Hermitian LU QR echelon
Systems involving triangular matrices are often easy to solve. Try to rewrite a as a product that contains a triangular seems like a good idea. One way is using LU- where PA = LU where P is a permutation, L is a lower- and U is an upper triangular. There is also the, A = GG H, where A is Hermitian and positive-definite and G is lower triangular. Hermitian LU QR echelon
Consider the equation Ax = y. If a can be factorized, A = GG H, this equation can be turned into two new equations: { Gz = y G H x = z both of these equations are easy to solve. Hermitian LU QR echelon
Calculating the Looking at the relation A = LL for a real positive definite 3 3 we get: L 1,1 0 0 A = L 2,1 L 2,2 0 L 3,1 L 3,2 L 3,3 L 1,1 L 2,1 L 3,1 0 L 2,2 L 3,2 0 0 L 3,3 L 2 1,1 L 2,1 L 1,1 L 3,1 L 1,1 = L 2,1 L 1,1 L 2 2,1 + L 2 2,2 L 3,1 L 2,1 + L 3,2 L 2,2 L 3,1 L 1,1 L 3,1 L 2,1 + L 3,2 L 2,2 L 2 3,1 + L 2 3,2 + L 2 3,3 Hermitian LU QR echelon
Calculating the Since A is symmetric we only need to calculate the lower triangular part. L 2 1,1 L 2,1 L 1,1 L 2 2,1 + L 2 2,2 L 3,1 L 1,1 L 3,1 L 2,1 + L 3,2 L 2,2 L 2 3,1 + L 2 3,2 + L 2 3,3 For the elements L i,j we get: j 1 L j,j = Aj,j ( L i,j = 1 A i,j L j,j k=1 L 2 j,k ) j 1 L i,k L j,k, i > j k=1 We notice that we only need the elements above and to the left to calculate the next element. Hermitian LU QR echelon
Applications of Are there any interesting matrices that can be easy factorized? Any covariance is positive-definite and any covariance based on measured data is going to be symmetric and real-valued. From the last two it follows that this is Hermitian. Example application: generating variates according to a multivariate distribution with covariance Σ and expected value µ Using the you get the simple ula X = µ + G Z where X is the variate, Σ = GG H and Z is a vector of standard normal variates. Hermitian LU QR echelon
QR Theorem Every n m A have a decomposition where A = QR R is a n m upper triangular.. Q is a n n unitary. Hermitian LU QR echelon
QR Given a QR- we can solve a linear system Ax = b by solving Rx = Q 1 b = Q H b. Which is can be done fast since R is a triangular. QR- can also used in solving the linear least square problem. It plays an important role in the QR-method used to calculate eigenvalues of a numerically. Hermitian LU QR echelon
A canonical is a standard way of describing an object. There can be several different kinds of canonical for an object. Some examples for matrices: Diagonal (for diagonalizable matrices) echelon (for all matrices) Jordan canonical (for square matrices) (for all matrices) Hermitian LU QR echelon
echelon Definition A is written on reduced row echelon when they are written on echelon and their pivot elements are all equal to one and all other elements in a pivot column are zero. B = 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Theorem All matrices are similar to some reduced row echelon. Hermitian LU QR echelon
Definition (Jordan block) A Jordan block is a square of the λ 1 0... 0 0 λ 1... 0 J m (λ) =......... 0 0... λ 1 0 0... 0 λ Hermitian LU QR echelon
Definition (Jordan ) A Jordan is a square of the J m1 (λ 1 ) 0... 0 0 J m2 (λ 2 )... 0 J =...... 0 0... J m1 (λ k ) Hermitian LU QR echelon
Theorem All square matrices are similar to a Jordan. The Jordan is unique except for the order of the Jordan blocks. This Jordan is called the of the. Theorem (Some other interesting of the Jordan normal ) Let A = S 1 JS a) The eigenvalues of J is the same as the diagonal elements of J. b) J has one eigenvector per Jordan block. c) The rank of J is equal to the number of Jordan blocks. d) The normal is sensitive to perturbations. This means that a small change in the normal can mean a large change in the A and vice versa. Hermitian LU QR echelon
Theorem All A M m n can be factorized as A = UΣV H where U and V are unitary matrices and [ ] Sr 0 Σ = 0 0 where S r is a diagonal with r = rank(a). The diagonal elements of S r are called the singular values. The singular values are uniquely determined by the A (but not necessarily their order). Hermitian LU QR echelon
Very often referred to as the SVD (singular value decomposition). Used a lot in statistics and ination processing. Can be used to quantify many different qualities of matrices, more on this in later lectures. Hermitian LU QR echelon
In everyday language two matrices are similar if they have almost the same elements or structure. But there is also a precise mathematical relation between two matrices that is called similar. Definition Two matrices, A and B, are similar if A = S 1 BS. Hermitian LU QR echelon
Interesting of similar matrices share several : Eigenvalues (but generally not eigenvectors) Determinant Trace We have already seen some examples of why similar matrices are interesting: Diagonalizable matrices A = S 1 BS Permutation matrices A = PBP A = S 1 JS Similarity between matrices mean they represent the same linear mapping described in different basis. Hermitian LU QR echelon
and matrices Hermitian matrices matrices Hermitian LU QR echelon
QΛQ 1 LU- GG H QR- CF Jordan canonical S 1 JS UΣV H Hermitian LU QR echelon