Lecture 15 Review of Matrix Theory III Dr. Radhakant Padhi Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore
Matrix An m n matrix is a rectangular or square array of elements with m rows and n columns. Eg: A m n a a a a a a 11 12 1n 21 12 2n = a a a m1 m2 mn For each subscipt, a, i = the row, and j = the column. ij If m = n, then the matrix is said to be a "square matrix". 2
Vector If a matrix has just one row, it is called a "row vector" Eg. [ b b b ] 11 12 1n If a matrix has just one column, it is called a "column vector" Eg. c c c 11 21 m1 3
Null Matrix / Diagonal Matrix Null matrix : A matrix with all zero elements 0 0 0 0 0 0 0 0 Eg. A = 0 0 0 0 Diagonal Matrix : A square matrix with all off diagonal elments being zero a11 0 0 0 0 a22 0 0 Eg. A = 0 0 a33 0 0 0 0 0 4
Linear Dependence/Independence A set of n column vectors s1, s2,, sn, is said to be "linearly dependent" if there exist constants α1, α2,, αn, not all zero, such that α s + α s + + α s = 0 1 1 2 2 If the above equation holds only when α1= α2 = = α n, then the vectors are "linearly independent". n n 5
Example: Linear Dependence/Independence Question : 3 [ ] T [ ] a [ ] Are the vectors a = 1 1 0, = 1 0 1, a = T 1 2 0 1 1 linearly independent? T Answer : We must test whether xa1 + ya2 + za3 = 0 admits any non-trivial solution. The determinant of the coefficient is 1 1 0 D = 1 0 1 = 2 0 0 1 1 Hence the vectors are linearly independent. 6
Rank of a Matrix The rank of a matrix A m n is equal to the number of linearly independent rows or columns. The rank can be found by finding the higest-order square submatrix that is non-singular. 1 5 2 Eg. A = 4 7 5 3 15 6 Here A = 0. So, the A matrix is singular and hence rank( A) 3 1 5 Choosing the sub-matrix B=, B 27 0 4 7 = Hence rank( A) = 2 7
Matrix Operations Addition The sum of two matrices, written A + B = C, is defined 2-1 7-5 9-6 by aij + bij = cij Eg. 3 5 + -4 3 = -1 8 Subtraction The difference between two matrices, written A B = C, is defined 2-1 7-5 -5 4 by aij bij = cij Eg. 3 5 = -4 3 7 2 8
Matrix Operations Multiplication The product of two matrices, written AB = C, is defined by c = a b a a a b b b 11 12 13 11 12 13 Eg. A = ; B b21 b22 b 23 a21 a22 a = 23 b31 b32 b33 AB = n ij ik jk k = 1 ab 11 11 + ab 12 21 + ab 13 31 ab 11 12 + a b + a b a b + a b + a b a b + a b + a b a b + a b + a b a b + a b + a b 12 22 13 32 11 13 12 23 13 33 21 11 22 21 23 31 21 12 22 22 23 32 21 13 22 23 23 33 Schur / Hadamard / Direct product AB i = aij ib ij, if A& Bhave some dimension 9
Matrix Identities Commutative Law Associative Law Transpose of sum Transpose of product A + B = B + A AB BA Α + B + C = A + B + C ( ) ( ) Α BC = ( ) ( ) ( ) ( ) AB C T T T A + B = A + B AB T T T T A = AB AB = B A A A BA Determinant identities det det det = det det det = det B 10
Trace Trace of T r Α n n Α = ( ) n i= 1 a ij : Sum of the diagonal elements 1 3 Eg A =, Tr( Α ) = 1 + 4 = 5 2 4 11
Example: Eigenvalues/Eigenvectors 1 3 Question : Find eigenvalues/eigenvecotrs of A = 2 2 Solution : Eigenvalues: The characteristic equation is given by 1 λ 3 det ( A λi2 ) = = 0 2 2 λ 2 ( λ)( λ) λ λ ( λ )( λ ) 1 2 6= 3 4= 4 + 1 = 0 Hence the eigenvalues are λ = 1, λ = 4 1 2 12
Example: Eigenvalues/Eigenvectors Eigenvectors: a1 Let X ( ) 1 =. Then the equation A 1I2 X1 0 leads to: b λ = 1 1 4 3 a1 0 3a1 + 3b1 = 0 ( a ) 1 b1 2 2 4b = 1 0 = 2a1 2b1 = 0 The solutions can be expressed as a1 = b1 = α for any α. If we put α = 1, then an eigenvector is 1 X1 = 1 13
Example: Eigenvalues/Eigenvectors Similarly ( A λ I ) X = 0 implies 2 2 2 1+ 1 3 a 2a + 3b = 0 = 0, or 2 2 2 2 2 1 b + 2 2a2 + 3b2 = 0 The eigenvectors for this case are β X 2 = 2 β 3 for any nonzero β. Assuming β = 3 gives the eigenvector 3 X 2 = 2 14
Example: Eigenvalues/Eigenvectors 15
Example: Eigenvalues/Eigenvectors 16
Example: Eigenvalues/Eigenvectors 17
Additional Properties of Eigenvalues/Eigenvectors Α= λ1 λn Tr Α = λ + + λ ( ) If det 1 λi Α = λ + a λ + + aλ + a ( ) n 1 n n 1 1 0 ( ) ( ) n n 1 = Α 0 = Α then a Tr, a 1 If Α is Real and Skew-Symmetric n n n ( T Α= Α ) ( Α= Α ) or Complex and Skew-Hermition, Then its eigenvalues are all pure imaginary 18
Additional Properties of Eigenvalues/Eigenvectors [ ] If Α n nis Real and Symmetric or Complex and Hermition, then its eigenvalues are all real. If A is a symmetrical matrix, then the eigenvectors associated with two distirct eigenvalues are orthogonal If An nis a symmetric matrix, then T X ΑX λ ( ) ( ) min < R X < λmax,where, R X, X 0 T X X 19
Proof 20
Proof 21
Proof 22
Proof 23
Generalized Eigenvectors Suppose A has eigenvalues λ, λ, λ 3 3 1 2 2 Let A A+ εb, ε << 1 ε such that Α has distinct eigenvalues λ, λ, λ ε where λ = λ + δ, δ << 1 3 2 1 2 3 and the corresponding eigenvectors ν1, ν2, ν3 = ν2 + δw Then as ε 0, the following equations are satisfied: ( λi ) νi ( A λ2i) w= ν2 { ν ν w} (1) A I = 0 i= 1, 2 (2) Note that,, are linearly independent 1 2 and w is called a "generalited eigenvector" of A. 24
Quadratic Forms T = [ ] Suppose X x1, x2,, x n. Then any polynomial function the elements in which every term is of degree two is known as a quadratic form. Thus, if n = 3, then 2 2 2 x1 + 8x1x2 + x2 + 6x2x3 + x3 is an example of a quadratic form. Quadratic forms can always be expressed as a matrix product T of the form X AX Utility: Optimization theory, Optimal control theory etc. 25
Example: Quadratic Form The example above can be written as 1 4 0 x1 x1 x2 x 3 4 1 3 x 2 0 3 1x [ ] 3 In this representation, A is required to be symmetric matrix, Nonsymmmetric representations are possible with 1 0 0 Eg: A = 8 1 2 0 4 1 but the symmetric form is "standard" 26
Example: Singular Value Decomposition 2 0 2 1 0 1, T A= A A 0 0 0 1 0 1 = 2 0 2 λ = 4,0,0 1,2,3 ( T AA) 1 0 1 1 1 X1 = 0, X 2 1, X 3 0 2 = = 2 1 0 1 for λ = 4 for λ = 0 for λ = 0 ( ) ( ) ( ) 27
Example: Singular Value Decomposition σ1 λ 1 2 σ 2 λ 2 0 = = Note: The values are ordered σ 3 λ 0 3 1 1 1 1 0 1 1 1 1 Y1 = ( AX1) = 0 σ1 2 1 0 1 2 = 2 1 1 1 2 [ ] ( ) T 2 1 2 Next, find Y = a b such that Y and Y will be orthonormal i.e. Y, Y = Y T T 2 2 1 Y2 = a b= 0 and Y2, Y2 = Y2 Y2 = a + b = 1 28
Example: Singular Value Decomposition Solution: a b 1/ 2. Hence, Y P 1 2 = = = = 2 = Y Y 1 1 1 2 1 1 1 0 1 T X1 X2 X3 1 = = 0 2 0 1 1 2 1 Q 2 1 0 1 σ 1 0 0 2 0 0 D = 0 0 0 = 0 0 0 1 0 1 1 1 1 2 0 0 1 1 0 1 Verify : PDQ = 0 2 0 2 1 1 0 0 0 = 2 1 0 1 = 1 0 1 A 29
1 Derivative of A ( t) Let Hence ( ) ( ); Then ( ) ( ) 1 A t = B t A t B t = I d dt () () ( At Bt ) () At () db t dt db t dt = 0 () da() t = () () () + Bt = dt da t () 1 A t B t 1 da () t ( ) 1 da t 1 dt dt = A () t A t dt () 0 30
Practice Problems 2 1 1. Let A=. Find eigenvalues/eigenvectors of. In addition, find 1 2 A the eigenvalues/eigenvectors of A, A m= 3,4 and I + 2A+ 4 A without computing these matrices. 2. Proove ( AB) 1 1 1 = B A ( ) 1 m 2,if the indicated matrix inverses exist. 3. An n n Hadamard matrix A has all elements that are ( ) T ± 1 and satisfies A A= ni. Show that det A = n 4. Show that the determinant of a negative definite n n matrix is positive if n is even and negative if n is odd. 5. Let A and P be n n matrices with P nonsingular. Show that 1 ( ) = Tr ( P AP) Tr A n/2 31
Practice Problems 6. Following the standard definitions, Show that for a fixed X n R X X as p 1 2 ( ) 7. Compute A, A, A and p A of the matrix in Problem-1 8. Find the singular value decomposition of ( ( )) 1 0 1 A = 0 0 1. Give all the steps. Verify the results using MATLAB. 1 T 9. Show that X AX = AX X 2 n n 10. Show that if F f X R, X R, F F F then = X p n f X T m n p 32
33
34