1. Elements of linear algebra

Transcription

1 Elements of linear algebra Contents Solving systems of linear equations 2 Diagonal form of a square matrix 3 The Jordan normal form of a square matrix 4 The Gram-Schmidt orthogonalization process 5 The matrix exponential function Solving systems of linear equations Consider the following system of linear equations: a x + a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2 a m x + a m2 x a mn x n = b m () Here a ij R, b j R The system () can be written in the form A X = B (2) where A = a a 2 a n a 2 a 22 a 2n a m a m2 a mn, X = x x 2 x n, B = b b 2 b m It is well known that if the system () or (2) is consistent, that is, there is a vector (a particular solution) X () = x () x () 2 x () n, satisfying (2), then each solution X of (2) can be written in the form X = X () + t X () + t 2 X (2) + + t r X (r) (3) Here t R,, t r R are free variables and {X (), X (2),, X (r) } is a fundamental system of solutions for the homogeneous system X (j) = x (j) x (j) 2 x (j) n A X =, (4), j =,, r

2 Thus, the general solution of () can be represented as follows x = x () + t x () + t 2 x (2) + + t r x (r) x 2 = x () 2 + t x () 2 + t 2 x (2) t r x (r) 2 x n = x () n + t x () n + t 2 x (2) n + + t r x (r) n (5) The solution (5) can be found, for instance, due to Gauss-Jordan algorithm Example x +2 x 2 +3 x 3 = 4x +5 x 2 +6 x 3 = 7x +8 x 2 +9 x 3 = 2 The system (6) can be expressed in the matrix form as follows (7) Using the Gauss-Jordan algorithm one can reduce (7) first to the row echelon form ( ) and then to the reduced row echelon form ( 2 28/3 29/3 (6) (8) ) (9) The variable x 3 can be regarded as a free variable, x 3 = t It follows from (9) that is the solution of (6) x = t x 2 = t x 3 = t > restart; with(linearalgebra): > A := Matrix([[,2,3],[4,5,6],[7,8,9]]); > GaussianElimination(A); > ReducedRowEchelonForm(<A>); A :=

3 > B:=Vector([,,2]); 2 B := > X:=LinearSolve(A, B,free= t ); # the general solution t 3 X := t 3 2 t 3 Example 2 x +2 x 2 +3 x 3 = 4x +5 x 2 +6 x 3 = 7x +8 x 2 +9 x 3 = 3 If we try to solve this system with Maple we obtain the following result > with(linearalgebra): > A := Matrix([[,2,3],[4,5,6],[7,8,9]]); > B:=Vector([,,3]); A := > X:=LinearSolve(A, B,free= t ); B := 3 () Error, (in LinearAlgebra:-LA_Main:-LinearSolve) inconsistent system Thus, the system () is inconsistent 2 Diagonal form of a square matrix Consider a matrix A = (a ij ) n n which is a square n n matrix with real (or complex) entries The main objective is to reduce this matrix to a diagonal form It means that one should find a square non-degenerate n n matrix C, a transition matrix, and a matrix D for which the equality C A C = D () 3

4 holds Here D is a diagonal matrix, D = λ λ 2 λ n Remark Note that the problem of diagonalization can have no solution even if we consider the complex matrices For instance, the matrix ( ) M = is not diagonalizable In general, there is a matrix equality of type () with a matrix D having Jordan normal form The diagonal form is a particular case of Jordan normal form Eigenvectors and eigenvalues of a square matrix Let us suppose that matrices C and D exist for a given matrix A In order to obtain the diagonal matrix D we should find all eigenvalues of A Then matrix C can be determined in terms of eigenvectors of A (see the general case below) Definition A non-trivial vector X = x x 2 x n, X, is called eigenvector of a square n n matrix A if there is a number λ C such that the equality holds In this case λ is called eigenvalue of A Note that the equality (2) can be rewritten in the form A X = λ X (2) (A λe) X = (3) where E is the identity matrix The matrix equality (3) can be regarded as a homogeneous system of linear equations It is well known that it has a non-trivial solution if and only if det(a λe) = Recall that the polynomial χ A (λ) = det(a λe) is called characteristic polynomial of a matrix A Thus, a complex number λ is an eigenvalue of a matrix A if and only if λ is a root of characteristic polynomial of A Example 3 Find the eigenvalues of the matrix A =

5 The characteristic polynomial 5 λ 8 6 det 8 25 λ 46 = 6 + 5λ + 2λ 2 λ λ The roots of the characteristic polynomial, the eigenvalues of A, are equal to 2,, 3 Remark The diagonal entries of the matrix D in () are eigenvalues of A In view of () there exists a non-degenerate matrix C such that C C = It can be showed that as rows of such a transition matrix C the corresponding eigenvectors can be chosen (see the general case below) > restart; with(linearalgebra): > A := Matrix([[-5,-8,-6],[-8,-25,-46],[2,7,32]]); A := > CharacteristicPolynomial(A,lambda); λ 2 λ 2 + λ 3 > solve(%,lambda);, 2, 3 > L:=Eigenvalues(A,output= list ); L := [, 2, 3] > E:=Matrix(3,3,shape=identity); #The identity matrix E := > B:=Vector(3,shape=zero); B := > X:=LinearSolve(A-L[2]*E,B,free= t ); > #this is eigenvector for the > eigenvalue L[2]=-2 X := 2 t 3 t 3 > X2:=LinearSolve(A-L[]*E,B,free= t ); > #this is eigenvector for the > eigenvalue L[]= 5

6 X2 := 4 t 3 t 3 > X3:=LinearSolve(A-L[3]*E,B,free= t ); > #this is eigenvector for the > eigenvalue L[3]=3 > t[3]:=; X3 := t 3 := t 3 t 3 t 3 t 3 > C:=Matrix([X,X2,X3]); # A transition matrix 4 C := 2 > Diag:=C^(-)AC; # this is to check the result 2 Diag := 3 Thus, we have obtained a transition matrix 4 C = 2 Remark The characteristic polynomial is defined up to sign ± The diagonal form and the transition matrix in general case Suppose that A is a diagonalizable matrix and let χ A (λ) = det(a λe) = ±(λ λ ) k (λ λ s ) ks (4) be a decomposition of characteristic polynomial of A over C The eigenvalues λ,, λ s are pairwise distinct and k + + k s = n where n is the dimension of A The following theorem provides a necesary and sufficient condition for the existence of diagonal form of A Theorem In previous notation suppose that for each l =,, s the system (A λ l E)X = has k l linearly independent solutions X l,,, X l, kl to the eigenvalue λ l Then C A C = D where which are eigenvectors of A corresponding 6

7 D = λ λ λ s is a diagonal matrix with eigenvalues of A as diagonal entries (λ l appears k l times) and the columns of C are linearly independent eigenvectors X,,, X, k,, X s,,, X s, ks In particular, if the characteristic polynomial of A has no multiple roots (ie all k l = in (4), the total number of different eigenvalues is equal to n) then the diagonal form of A exists Example 4 Let us find the transition matrix C in the equality C C = of Example 3 For the eigenvalue λ = 2 we obtain an eigenvector X = λ 2 = an eigenvector X 2 = X 3 = Then 4 λ s 2, for the eigenvalue and finally, for the eigenvalue λ 3 = 3 an eigenvector C = 4 2 > restart; with(linearalgebra): > A := «-5,-8,2> <-8,-25,7> <-6,-46,32»; A := > K:=CharacteristicPolynomial(A,lambda); K := 6 5 λ 2 λ 2 + λ 3 > factor(k); (λ ) (λ 3) (λ + 2) > (lambda,c):=eigenvectors(a); 7

8 > C; λ, C := 3 2, > C^(-); #this is the inverse matrix > Diag:=C^(-)AC; 3 Diag := 2 3 The Jordan normal form of a square matrix Sometimes the characteristic polynomial of a square matrix A has multiple roots and A has no diagonal form However, instead of diagonal form there is a Jordan normal form of A, that is, there is a matrix equality C A C = J with a block diagonal matrix J and some non-degenerate matrix C The matrix J is of block diagonal type J m (λ ) J mt(λ ) (5) J mj (λ s ) J mk (λ s) where λ λ λ J p (λ) = λ λ is a p p matrix with entries equal to λ on the diagonal and entries equal to just immediately over the diagonal The other entries are trivial J p (λ) is called a Jordan cell 8

9 In the form (5) the number and the dimensions of Jordan cells for each eigenvalue λ k are uniquely defined by the matrix A Remark The diagonal form is a particular case of Jordan form In the diagonal form all Jordan cells are matrices Example 5 Let us find the Jordan form of a matrix A = The characteristic polynomial is equal (up to the sign) to The Jordan form (λ + )(λ 3) 2 J = 3 3 has one -cell J ( ) and one 2-cell J 2 (3) Note that A has no diagonal form > with(linearalgebra): > A:=Matrix([[23,28,52],[,6,27],[-4,-9,-34]]); A := > CharacteristicPolynomial(A,lambda); > factor(%); > Eigenvalues(A); > J:=JordanForm(A); λ 5 λ 2 + λ 3 (λ + ) (λ 3) 2 J := The Gram-Schmidt orthogonalization process Consider the euclidian space R n, that is, the vector space R n equipped with the inner product The inner product two vectors X = (x,, x n ), Y = (y,, y n ) 9

10 is defined as follows X Y = x y + + x n y n Recall that the euclidian norm (the length) X of a vector X is the following number X = X X = x x 2 n Definition A vector X R n is called normalized if X = Definition Two vectors X R n and Y R n are called orthogonal if X Y = A system of vectors S is called orthogonal if all vectors of the system are pairwise orthogonal A system of vectors S is called orthonormal if it is orthogonal and each vector in S is normalized Note that any orthogonal system consists of linearly independent vectors In linear algebra the following orthogonalization problem is often needed to be solved Orthogonalization problem For a given system of vectors to find an orthogonal (orthonormal) system S = {V,, V k }, V i R n, Σ = {U,, U m }, m k, such that the linear subspaces spanned by S and Σ coincide: V,, V k = U,, U m This problem can be solved with the help of the Gram-Schmidt orthogonalization process The Gram-Schmidt orthogonalization process This process works inductively Without loss of generality we may assume that there is no trivial vector in S Step Let U = V Step 2 It follows from the first step that V = U We will seek a vector U 2 in the form The condition U 2 U = is required and consequently, It follows that U 2 = V 2 a 2 U (6) = U 2 U = (V 2 a 2 U ) U = V 2 U a 2 U U a 2 = V 2 U U U, U and U 2 are orthogonal If U 2 then it is included in the system Σ (otherwise we omit trivial vector) Now we go on to the next step Step 3 Due to (6) V, V 2 = U, U 2 We will seek a vector U 3 in the form U 3 = V 3 a 3 U a 32 U 2 (7)

11 The conditions U 3 U = and U 3 U 2 = are required and consequently, = U 3 U = (V 3 a 3 U a 32 U 2 ) U = V 3 U a 3 U U, = U 3 U 2 = (V 3 a 3 U a 32 U 2 ) U = V 3 U 2 a 32 U 2 U 2 since U U 2 = U 2 U = It follows that a 3 = V 3 U U U, a 32 = V 3 U 2 U 2 U 2 and U, U 2, U 3 are pairwise orthogonal If U 3 then it is included in the system Σ and the process is going on Step l Suppose that we have already determined pairwise orthogonal vectors U, U 2,, U s such that V, V 2, V l = U, U 2,, U s where Then as above U s = V l a l, U a l,s U s (8) a l,j = V l U j U j U j (9) If U s then it is included in Σ and so on Since the number of vectors is finite then this process will terminate after k steps and we obtain a desired orthogonal system Σ The following transform (normalization) is to be done in order to make Σ orthonormal: {U,, U m } { U U,, } U m U m The Gram-Schmidt orthogonalization process from a geometrical point of view The l-th step of the Gram-Schmidt orthogonalization process can be viewed as follows (see Figure below) Denote L s = U, U 2,, U s A given vector V l should be represented as the (unique) sum V l = U s + (V l U s ) where U s is orthogonal to the subspace L s (the vector U s is called the orthogonal component of V l with respect to L s ) and V l U s L s (the vector V l U s is the orthogonal projection of V l onto L s ) It is sufficient to determine (cf (8)) since V l U s = a l, U + + a l,s U s (2) U s = V l (V l U s ) The coefficients a l,j in (2) are defined by the formula (9)

12 U s V l L s = U, U 2,, U s V l U s Fig Note that the distance between the endpoint of V l and the subspace L s is equal to the length U s Example 6 Determine the orthogonal basis of the subspace L R 4 spanned by the vectors V = (, 2, 2, ), V 2 = (,, 5, 3), V 3 = (4, 5, 3, 8) Solution Apply the Gram-Schmidt orthogonalization process to the system S = {V, V 2, V 3 } Step Step 2 U = V = (, 2, 2, ) where Consequently, Step 3 a 2 = V 2 U U U = U 2 = V 2 a 2 U ( 5) + ( ) ( ) 2 = U 2 = V 2 ( )U = (2, 3, 3, 2) where Hence, U 3 = V 3 a 3 U a 32 U 2 a 3 = V 3 U U U = 2, a 32 = V 3 U 2 U 2 U 2 = 3 U 3 = V 3 ( 2)U 3U 2 = Thus, U 3 is not included in the orthogonal basis and the subspace L is 2-dimensional The orthogonal basis is Σ = {U, U 2 } The normalized system Σ norm is { } {( U U, U 2 =, U 2 > with(linearalgebra): 2, 2, ) ( 2,, , 3, )} 2 26

13 > v:=vector[row]([,2,2,-]); v := [, 2, 2, ] > v2:=vector[row]([,,-5,3]); v2 := [,, 5, 3] > v3:=vector[row]([4,5,-3,8]); v3 := [4, 5, 3, 8] > Basis([v,v2,v3]); #this is to determine > the dimension of subspace <v,v2,v3> [[, 2, 2, ], [,, 5, 3]] > res:=gramschmidt([v,v2,v3]); res := [[, 2, 2, ], [2, 3, 3, 2]] > A:=Matrix([[res[]],[res[2]]]); [ ] 2 2 A := > C:=ATranspose(A); # this must be the > diagonal matrix of inner products C := [ 26 > Normalize(res[],2); # 2 stands for the euclidian norm [ ],,, 5 5 > Normalize(res[2],2); [ ] , 26, 26, > res_norm:=gramschmidt([v,v2,v3],normalized); res_norm := [[ ],,,, 5 5 ] [ ]] , 26, 26, Example 7 Determine the distance d between the (endpoint of the) vector X =(, 5,, ) and the subspace L R 4 given as the subspace of solutions of the linear homogeneous system { 4x L : 2 x 3 + 3x 4 = 2x + 2x 2 + x 3 + x 4 = Solution First of all, let us find a basis of L As a basis a fundamental system of solutions of the above linear system can be taken > restart;with(linearalgebra): > A := Matrix([[,4,-,3],[2,2,,]]); [ 4 3 A := 2 2 > B:=Vector([,]); ] 3

14 [ B := > X:=LinearSolve(A, B,free= t ); > t[2]:=;t[4]:=; X := ] 3 t 2 2 t 4 t 2 4 t t 4 t 4 t 2 := t 4 := > v:=transpose(x); > t[2]:=;t[4]:=; > v2:=transpose(x); v := [ 2,, 3, ] t 2 := t 4 := v2 := [ 3,, 4, ] Thus, the vectors V = ( 2,, 3, ) and V 2 = ( 3,, 4, ) form a basis of L The second step is the orthogonalization of the basis S = {V, V 2 } We obtain the orthogonal basis {U, U 2 } of L where U = ( 2,, 3, ), U 2 = ( 3, 7,, 9) The last step is the orthogonalization of the system S = {U, U 2, X} (see Figure There the vector V l should be thought as X, the vector U s corresponds to the resulting vector Y below) As a result we get the orthogonal system {U, U 2, Y } where ( 3 Y = 5, 3 5, 5, 9 ) 5 Consequently, > with(linearalgebra): d = Y = = 65 5 > v:=vector[row]([-2,,3,]); v := [ 2,, 3, ] > v2:=vector[row]([-3,,4,]); v2 := [ 3,, 4, ] > U:=GramSchmidt([v,v2]); > u:=u[]; U := [ [ 2,, 3, ], 4 [ 3 7,, 7, 9 ]] 7

15 u := [ 2,, 3, ] > u2:=7*u[2]; # we scale the vector U[2] by a non-trivial factor u2 := [ 3, 7,, 9] > X:=Vector[row]([-,5,,-]); X := [, 5,, ] > U:=GramSchmidt([u,u2,X]); > Y:=U[3]; U := > distance:=norm(y,2); [ [ 2,, 3, ], [ 3, 7,, 9], Y := [ 3 5, 3 5, 5, 9 ] 5 distance := [ 3 5, 3 5, 5, 9 ]] 5 Remark For this example we split the whole Maple worksheet into two parts according to the algorithm It is more convenient to compute the distance using a single worksheet 5 The matrix exponential function Given a square matrix A with real or complex entries the matrix exponential function exp is defined as follows exp(a) = E + A + A2 2! + A3 3! + + An n! + = n= A n n! (2) Here E stands for the identity matrix The matrix series converges to the matrix exp(a) for any matrix A The exponential matrix plays a big role in solving linear systems of differential equations Example 8 Determine exp(a) if A = λ λ 2 λ 3 Solution Since λ n A n = λ n 2 λ n 3 and we have the expansion series for any number z C e z = exp z = + z + z2 2! + z3 3! + + zn n! + = z n n!, then it follows from the definition (2) that e λ exp(a) = e λ 2 e λ 3 5 n=

16 Example 9 Determine exp(ta) (here t R) if ( ) A = > with(linearalgebra): > A:=Matrix([[,],[-,]]); [ A := > with(linalg): exponential(a*t); [ e t cos(t) e t sin(t) e t sin(t) e t cos(t) As a result we obtain the following matrix ( e exp(ta) = t cos t e t sin t e t sin t e t cos t Example Determine exp(tj) if J = ] λ λ λ λ ] ) > with(linearalgebra): > J:=Matrix([[lambda,,,],[,lambda,,],[,,lambda,],[,,,lambda > ]]); J := λ λ λ λ > with(linalg): exponential(j*t); e (t λ) t e (t λ) 2 t2 e (t λ) 6 t3 (t λ) e e (t λ) t e (t λ) 2 t2 (t λ) e e (t λ) (t λ) t e (t λ) e Thus, exp(tj) = e tλ t e tλ t 2 2! etλ t 3 3! etλ e tλ t e tλ t 2 2! etλ e tλ t e tλ e tλ 6