Algebra II. Paulius Drungilas and Jonas Jankauskas

Algebra II Paulius Drungilas and Jonas Jankauskas Contents 1. Quadratic forms 3 What is quadratic form? 3 Change of variables. 3 Equivalence of quadratic forms. 4 Canonical form. 4 Normal form. 7 Positive definite quadratic forms. 9 Sylvester s criterion. 9 Exercises 10 2. Euclidean space 13 Euclidean space 13 Component and projection 16 Gram-Schmidt orthogonalization process 17 Orthogonal complement 18 Finding the orthogonal complement 19 Finding the projection 20 Exercises 21 3. Linear maps 24 What is linear map? 24 What is matrix of linear map? 25 Change of basis 30 Important properties of matrices 31 Exercises 32 4. Eigenvalues and eigenvectors 38 1

What is eigenvalue and eigenvector? 38 How to find eigenvalues? 39 How to find eigenvectors? 40 Exercises 44 References 46 2

1. Quadratic forms 3 What is quadratic form? Let k be a field and let X := (x 1, x 2,..., x n ) be the vector (1 n matrix) of independent variables x 1, x 2,..., x n. We say that a polynomial f(x 1, x 2,..., x n ) k[x 1, x 2,..., x n ] is a quadratic form if there exists an n n symmetric matrix A = (a ij ), a ij k, such that n n f(x 1, x 2,..., x n ) = XAX t = a ij x i x j = i=1 j=1 n n 1 a ii x 2 i + 2 i=1 n i=1 j=i+1 a ij x i x j. Hence a quadratic form is a homogenious polynomial of degree two. We also say that A is the matrix of the quadratic form f(x 1, x 2,..., x n ). The rank of A is called the rank of the quadratic form f(x 1, x 2,..., x n ). Example 1. The matrix of the quadratic form f(x 1, x 2, x 3 ) = x 2 1 + 2x 2 2 x 2 3 + 4x 1 x 2 3x 1 x 3 is since 1 2 3/2 A = 2 2 0, 3/2 0 1 1 2 3/2 f(x 1, x 2, x 3 ) = XAX t = (x 1 x 2 x 3 ) 2 2 0 3/2 0 1 x 1 x 2 x 3. Change of variables. The linear transformation x 1 = c 11 y 1 + c 12 y 2 + + c 1n y n x 2 = c 21 y 1 + c 22 y 2 + + c 2n y n x n = c n1 y 1 + c n2 y 2 + + c nn y n, (1.1)

c ij k, is called a linear change of variables over the field k. In what follows, we shall say change of variables instead of linear change of variables over the field k. The matrix C = (c ij ) is called the matrix of the change of variables (1.1). Then the change of variables (1.1) can be rewritten as X = Y C t, where Y = (y 1, y 2,..., y n ). We say that the change of variables (1.1) is nonsingular if its matrix C is nonsingular, i. e. if det(c) 0. In what follows, we will consider only nonsingular changes of variables, so that change of variables means nonsingular change of variables, unless stated otherwise. The composition of two changes of variables X = Y C1 t and Y = ZC2 t is also a change of variables X = ZC2C t 1 t = Z(C 1 C 2 ) t whose matrix is C 1 C 2. Let f(x 1, x 2,..., x n ) be a quadratc form whose matrix is A. Consider a change of variables X = Y C. Then f(y C) = (Y C)A(Y C) t = Y (CAC t )Y t. Hence the matrix of the quadratic form f(y C) is CAC t. Equivalence of quadratic forms. Let X = (x 1, x 2,..., x n ) and Y = (y 1, y 2,..., y n ). We say that quadratic forms f(x 1, x 2,..., x n ) and g(y 1, y 2,..., y n ) are equivalent if there exists a change of variables X = Y C such that f(y C) = g(y ). Proposition 2. Equivalent quadratic forms have the same rank. A quadratic form f(x 1, x 2,..., x n ) is called canonical if its matrix is diagonal, i. e. if f(x 1, x 2,..., x n ) = a 1 x 2 1 + a 2 x 2 2 + + a n x 2 n. For example, the quadratic form f(x 1, x 2, x 3 ) = x 2 1 + 3x 2 2 5x 2 3 is canonical, whereas g(y 1, y 2, y 3 ) = 2y1 2 y2 2 + y 1 y 2 is not. Theorem 3. Every quadratic form over the field k whose characteristic char(k) 2 is equivalent to some canonical quadratic form. Canonical form. We say that a quadratic form g(y 1, y 2,..., y n ) is a canonical expression of a quadratic form f(x 1, x 2,..., x n ) if forms f and g are equivalent and g is canonical. In general canonical expression of a given quadratic form is not uniquely determined. Indeed, if g is a canonical expression of a quadratic form f then for any c k \ {0} the form c g is also a canonical expression of f. 4

Example 4. We will find a canonical expression g(y 1, y 2, y 3 ) of the quadratic form f(x 1, x 2, x 3 ) = x 2 1 x 2 2 11x 2 3 2x 1 x 2 + 4x 1 x 3 + 8x 2 x 3 and a change of variables X = Y C such that g(y ) = f(y C). We will find a form g by a procedure known as Lagrange s Reduction which consists essentially of repeated completing of the square. First of all, we collect all the terms with x 1 and complete the resulting expression to a square. f(x 1, x 2, x 3 ) = (x 2 1 2x 1 x 2 + 4x 1 x 3 ) x 2 2 11x 2 3 + 8x 2 x 3 = (x 2 1 + x 2 2 + 4x 2 3 2x 1 x 2 + 4x 1 x 3 4x 2 x 3 x 2 2 4x 2 3 + 4x 2 x 3 ) x 2 2 11x 2 3 + 8x 2 x 3 = (x 1 x 2 + 2x 3 ) 2 2x 2 2 15x 2 3 + 12x 2 x 3. The quadratic form 2x 2 2 15x 2 3 + 12x 2 x 3 is in two variables x 2 and x 3 and does not depend on x 1. Now we repeat the above described procedure for this quadratic form: f(x 1, x 2, x 3 ) = (x 1 x 2 + 2x 3 ) 2 2x 2 2 15x 2 3 + 12x 2 x 3 = (x 1 x 2 + 2x 3 ) 2 2(x 2 2 6x 2 x 3 ) 15x 2 3 = (x 1 x 2 + 2x 3 ) 2 5 2(x 2 2 6x 2 x 3 + 9x 2 3 9x 2 3) 15x 2 3 = (x 1 x 2 + 2x 3 ) 2 2(x 2 3x 3 ) 2 + 3x 2 3. Putting y 1 = x 1 x 2 + 2x 3 y 2 = x 2 3x 3 y 3 = x 3 we obtain the change of variables X = Y C, given by x 1 = y 1 + y 2 + y 3 x 2 = y 2 + 3y 3, x 3 = y 3 which transforms the quadratic form f(x 1, x 2, x 3 ) into its canonical expression g(y 1, y 2, y 3 ) = y 2 1 2y 2 2 + 3y 2 3.

Example 5. We will find a canonical expression g(y 1, y 2, y 3 ) of the quadratic form f(x 1, x 2, x 3 ) = x 1 x 2 + 4x 1 x 3 8x 2 x 3 and a change of variables X = Y C such that g(y ) = f(y C). The given quadratic form contains no squares of variables, therefore we consider an auxiliary change of variables x 1 = z 1 + z 2 x 2 = z 1 z 2. (1.2) x 3 = z 3 Then 6 f(z 1 + z 2, z 1 z 2, z 3 ) = (z 1 + z 2 )(z 1 z 2 ) + 4(z 1 + z 2 )z 3 8(z 1 z 2 )z 3 = z 2 1 z 2 2 4z 1 z 3 + 12z 2 z 3. Now the quadratic form z 2 1 z 2 2 4z 1 z 3 +12z 2 z 3 has a square z 2 1, and therefore we can proceed as in Example 4. f(z 1 + z 2, z 1 z 2, z 3 ) = z 2 1 z 2 2 4z 1 z 3 + 12z 2 z 3 = (z 2 1 4z 1 z 3 + 4z 2 3 4z 2 3) z 2 2 + 12z 2 z 3 = (z 1 2z 3 ) 2 (z 2 2 12z 2 z 3 ) 4z 2 3 = (z 1 2z 3 ) 2 (z 2 6z 3 ) 2 + 32z 2 3. Putting y 1 = z 1 2z 3 y 2 = z 2 6z 3 y 3 = z 3 we obtain the change of variables z 1 = y 1 +2y 3 z 2 = y 2 + 6y 3 (1.3) z 3 = y 3 which transforms the quadratic form f(z 1 + z 2, z 1 z 2, z 3 ) into its canonical expression g(y 1, y 2, y 3 ) = y 2 1 y 2 2 + 32y 2 3. Finally, from (1.2) and (1.3) we

7 obtain the change of variables X = Y C, given by x 1 = y 1 + y 2 + 8y 3 x 2 = y 1 y 2 4y 3, x 3 = y 3 which transforms f(x 1, x 2, x 3 ) into its canonical expression g(y 1, y 2, y 3 ) = y 2 1 y 2 2 + 32y 2 3. Normal form. Consider a quadratic form f(x 1, x 2,..., x n ) over the field k. If k = R (resp. k = C) then we say that f(x 1, x 2,..., x n ) is a real (resp. complex) quadratic form. Complex canonical quadratic form is called normal if all its coeficients belong to {0, 1}. Real canonical quadratic form is called normal if all its coeficients belong to { 1, 0, 1}. We say that a quadratic form (either real or complex) g(y 1, y 2,..., y n ) is a normal expression of a quadratic form f(x 1, x 2,..., x n ) if forms f and g are equivalent and g is normal. Theorem 6. Every complex quadratic form is equivalent to some normal quadratic form. Moreover, two complex quadratic forms are equivalent if and only if they have the same rank. A normal expression of a real quadratic form of rank r is a sum of r squares of distinct variables with coefficients ±1. Theorem 7. Every real quadratic form is equivalent to some normal quadratic form. Example 8. We will find a normal expression h(z 1, z 2, z 3 ) of the real quadratic form f(x 1, x 2, x 3 ) = x 2 1 x 2 2 11x 2 3 2x 1 x 2 +4x 1 x 3 +8x 2 x 3 and a change of variables X = ZC such that h(z) = f(zc). (Here X = (x 1, x 2, x 3 ) and Z = (z 1, z 2, z 3 ).) In Example 4 we obtained that the change of variables x 1 = y 1 + y 2 + y 3 x 2 = y 2 + 3y 3 (1.4) x 3 = y 3

8 transforms the form f(x 1, x 2, x 3 ) into its canonical expression g(y 1, y 2, y 3 ) = y 2 1 2y 2 2 + 3y 2 3. Note that Hence putting g(y 1, y 2, y 3 ) = y 2 1 ( 2y 2 ) 2 + ( 3y 3 ) 2. z 1 = y 1 z 2 = z 3 = 2y2 3y3, we obtain the change of variables y 1 = z 1 y 2 = 2 1 z 2 (1.5) 1 y 3 = 3 z 3 which transforms the quadratic form g(y 1, y 2, y 3 ) into its normal expression h(z 1, z 2, z 3 ) = z 2 1 z 2 2 + z 2 3. Finally, from (1.4) and (1.5) we obtain the change of variables X = ZC, given by x 1 = z 1 + 1 2 z 2 + 1 3 z 3 x 2 = 2 1 z 2 + 3z 3, 1 x 3 = 3 z 3 which transforms f(x 1, x 2, x 3 ) into its normal expression h(z 1, z 2, z 3 ) = z 2 1 z 2 2 + z 2 3. Theorem 9 (Sylvester s law of inertia). Let f(x 1, x 2,..., x n ) be an arbitrary real quadratic form. Then any two normal expressions of f have the same number of positive squares of variables. Let g(y 1, y 2,..., y n ) be a normal expression of a real quadratic form f(x 1, x 2,..., x n ). Denote by p(f) (resp. q(f)) the number of positive (resp. negative) squares of variables of the form g. The number p(f) (resp. q(f)) is called the positive index of inertia of f (resp. the negative index of

inertia of f) and the number s(f) := p(f) q(f) is called the signature of f. Note the sum p(f) + q(f) equals the rank of f. Theorem 10. Two real quadratic forms are equivalent if and only if they have the same rank and their signatures coincide. Positive definite quadratic forms. We say that a real quadratic form f(x 1, x 2,..., x n ) is positive definite if the inequality f(x 1, x 2,..., x n ) > 0 holds for any (x 1, x 2,..., x n ) R n \ {(0, 0,..., 0)}. Similarly, we say that f(x 1, x 2,..., x n ) is negative definite if the inequality f(x 1, x 2,..., x n ) < 0 holds for any (x 1, x 2,..., x n ) R n \ {(0, 0,..., 0)}. Note that a real quadratic form f(x 1, x 2,..., x n ) is negative definite if and only if the form f(x 1, x 2,..., x n ) is positive definite. Theorem 11. A real quadratic form f(x 1, x 2,..., x n ) is positive definite (resp. negative definite) if and only if p(f) = n (resp. q(f) = n). Sylvester s criterion. The leading principal minors of a symmetric matrix a 11 a 12 a 1n a 21 a 22 a 2n A = a n1 a n2 a nn are defined as a 11 a 12 a 1n a 1 := a 11, 2 := 11 a 12 a 21 a 22,..., a 21 a 22 a 2n n :=. a n1 a n2 a nn Theorem 12 (Sylvester s criterion). A real quadratic form f(x 1, x 2,..., x n ) is positive definite if and only if all the principal minors of its matrix are positive, i. e. if j > 0, j = 1, 2,..., n. 9

A real quadratic form f(x 1, x 2,..., x n ) is negative definite if and only if all the principal minors of its matrix satisfy ( 1) j j > 0, j = 1, 2,..., n. Example 13. We will show that the quadratic form f(x 1, x 2, x 3 ) = x 2 1 + 3x 2 2 + 39x 2 3 2x 1 x 2 + 4x 1 x 3 20x 2 x 3 is positive definite. Principal minors of the matrix of f(x 1, x 2, x 3 ) are 1 1 1 1 2 M 1 = 1, M 2 = 1 3 = 2, M 3 = 1 3 10 = 6. 2 10 39 Therefore, by Sylvester s criterion, the quadratic form f(x 1, x 2, x 3 ) is positive definite. 10 Exercises. Exercise 1. Find a canonical expression g(y ) of a given quadratic form f(x) and a change of variables X = Y C such that g(y ) = f(y C). a) f(x 1, x 2, x 3 ) = x 2 1 + 2x 2 2 + 7x 2 3 2x 1 x 2 + 4x 1 x 3 6x 2 x 3 ; b) f(x 1, x 2, x 3 ) = x 2 1 + 3x 2 2 + 25x 2 3 2x 1 x 2 + 4x 1 x 3 16x 2 x 3 ; c) f(x 1, x 2, x 3 ) = x 2 1 + 3x 2 2 + 22x 2 3 2x 1 x 2 + 4x 1 x 3 16x 2 x 3 ; d) f(x 1, x 2, x 3 ) = x 2 1 2x 2 2 27x 2 3 + 2x 1 x 2 4x 1 x 3 + 14x 2 x 3 ; e) f(x 1, x 2, x 3 ) = 2x 2 1 + 11x 2 2 + 96x 2 3 + 8x 1 x 2 + 12x 1 x 3 6x 2 x 3 ; f) f(x 1, x 2, x 3, x 4 ) = x 1 x 2 + x 3 x 4 ; g) f(x 1, x 2, x 3 ) = x 1 x 2 + x 2 x 3 ;

11 Answer: x 1 = z 1 + z 2 z 3 a)f K (z 1, z 2, z 3 ) = z1 2 + z2 2 + 2z3, 2 x 2 = z 2 + z 3 x 3 = z 3 x 1 = z 1 + z 2 + z 3 b)f K (z 1, z 2, z 3 ) = z1 2 + 2z2 2 + 3z3, 2 x 2 = z 2 + 3z 3 x 3 = z 3 x 1 = z 1 + z 2 + z 3 c)f K (z 1, z 2, z 3 ) = z1 2 + 2z2, 2 x 2 = z 2 + 3z 3 x 3 = z 3 x 1 = z 1 z 2 z 3 d)f K (z 1, z 2, z 3 ) = z1 2 3z2 2 4z3, 2 x 2 = z 2 + 3z 3 x 3 = z 3 x 1 = z 1 2z 2 13z 3 e)f K (z 1, z 2, z 3 ) = 2z1 2 + 3z2 2 + 3z3, 2 x 2 = z 2 + 5z 3 x 3 = z 3 f)f K (z 1, z 2, z 3, z 4 ) = z 2 1 z 2 2 + z 2 3 z 2 4, g)f K (z 1, z 2, z 3 ) = z 2 1 z 2 2, x 1 = z 1 + z 2 x 2 = z 1 z 2 x 1 = z 1 z 2 z 3 x 2 = z 1 + z 2 x 3 = z 3 x 3 = z 3 + z 4 x 4 = z 3 z 4 Exercise 2. Find normal expression of each of the quadratic form in Exercise 1 and apropriate changes of variables.

12 Exercise 3. Are the following quadratic forms equivalent? a) f(x 1, x 2, x 3 ) = x 2 1 + 2x 2 2 + 7x 2 3 2x 1 x 2 + 4x 1 x 3 6x 2 x 3, g(y 1, y 2, y 3 ) = y 2 1 + 3y 2 2 + 25y 2 3 2y 1 y 2 + 4y 1 y 3 16y 2 y 3 ; b) f(x 1, x 2, x 3 ) = x 2 1 + 7x 2 2 + 40x 2 3 4x 1 x 2 + 6x 1 x 3 + 6x 2 x 3, g(y 1, y 2, y 3 ) = 2y 2 1 + 5y 2 2 + 61y 2 3 4y 1 y 2 + 16y 1 y 3 + 2y 2 y 3 ; c) f(x 1, x 2, x 3 ) = x 2 1 + 2x 2 2 + 7x 2 3 2x 1 x 2 + 4x 1 x 3 6x 2 x 3, g(y 1, y 2, y 3 ) = y 2 1 + 4y 2 2 + 41y 2 3 2y 1 y 2 + 8y 1 y 3 + 10y 2 y 3 ; d) f(x 1, x 2, x 3 ) = x 2 1 + 2x 2 2 9x 2 3 2x 1 x 2 + 8x 1 x 3 26x 2 x 3, g(y 1, y 2, y 3 ) = 3y 2 1 + 7y 2 2 + 46y 2 3 6y 1 y 2 + 12y 1 y 3 + 12y 2 y 3 ; Answer: a) yes; b) yes; c) no; d) yes. Exercise 4. Is the following quadratic form positive definite? a) f(x 1, x 2, x 3 ) = x 2 1 + 2x 2 2 + 7x 2 3 2x 1 x 2 + 4x 1 x 3 6x 2 x 3 ; b) f(x 1, x 2, x 3 ) = x 2 1 + 3x 2 2 + 25x 2 3 2x 1 x 2 + 4x 1 x 3 16x 2 x 3 ; c) f(x 1, x 2, x 3 ) = x 2 1 + 3x 2 2 + 22x 2 3 2x 1 x 2 + 4x 1 x 3 16x 2 x 3 ; d) f(x 1, x 2, x 3 ) = x 2 1 2x 2 2 27x 2 3 + 2x 1 x 2 4x 1 x 3 + 14x 2 x 3 ; e) f(x 1, x 2, x 3 ) = 2x 2 1 + 11x 2 2 + 96x 2 3 + 8x 1 x 2 + 12x 1 x 3 6x 2 x 3 ; f) f(x 1, x 2, x 3, x 4 ) = x 1 x 2 + x 3 x 4. Answer: a) yes; b) yes; c) no; d) no; e) yes; f) no.

2. Euclidean space 13 Euclidean space. Let V be a vector space over the field of real numbers R. A map, : V V R is called an inner product of the space V if it satisfies the following conditions for all vectors u, v, v 1, v 2 V and any real number a R: 1) v 1 + v 2, u = v 1, u + v 2, u ; 2) v, u = u, v ; 3) a v, u = a v, u ; 4) v, v 0; 5) v, v = 0 if and only if v = 0. If, : V V R is an inner product of the space V then the pair (V,, ) is called Euclidean space. Example 5. Let u, v R n, u = (α 1, α 2,..., α n ), v = (β 1, β 2,..., β n ). Then the map, : R n R n R defined by u, v = α 1 β 1 + α 2 β 2 + + α n β n is an inner product of the vector space R n. Example 6. The ring of polynomials R[t] is a vector space over R. Then the map, : R[t] R[t] R defined by f(x), g(x) = 1 0 f(x) g(x) dx, f(x), g(x) R[t], is an inner product of the space R[t]. Example 7. Let V := R[t] and define the map, : V V R by f(x), g(x) = f(x) g(x)e x2 dx, where f(x), g(x) R[t]. Then the pair (R[t],, ) is an Euclidean space. Example 8. Denote by C([0, 1]) the set of all continuous real-valued functions on the interval [0, 1]. Then V :=C([0, 1]) is a vector space over R. Define the map, : V V R by f(x), g(x) = 1 0 f(x) g(x) dx,

where f(x), g(x) V. Then the pair (C([0, 1]),, ) is an Euclidean space. Example 9. Denote by M n (R) the set of n n matrixes whose coefficients are real numbers. Then M n (R) is a vector space over R and the map, : M n (R) M n (R) R defined by A, B = Tr(AB t ), A, B M n (R), is an inner product of the space M n (R). Here B t denotes the transpose of B and Tr(A) stands for the trace of A, i. e., if A = (a ij ) then Tr(A) = i a ii. Proposition 10. Suppose that V is a finite dimensional vector space over R. Then there exists an inner product of the space V. Example 11. Let (V,, ) be an Euclidean space and denote by O the zero vector of V. Then for any v V we have O, v = 0. Indeed, the first condition in the definition of an inner product implies and therefore O, v = 0. O, v = O + O, v = O, v + O, v, Theorem 12 (Cauchy-Bunyakovsky-Schwarz inequality). Suppose that (V,, ) is an Euclidean space. Then for all vectors u, v V, u, v 2 u, u v, v. Moreover, the equality holds if and only if vectors u and v are linearly dependent. Example 13. Let (R n,, ) be an Euclidean space as in Example 5. Cauchy- Bunyakovsky-Schwarz inequality implies 14 ( n ) 2 α i β i i=1 n i=1 α 2 i n i=1 β 2 i for any α i, β j R. Equality holds if and only if there exists t R such that either (α 1, α 2,..., α n ) = (tβ 1, tβ 2,..., tβ n ) or (β 1, β 2,..., β n ) = (tα 1, tα 2,..., tα n ).

Example 14. Let (R[x],, ) be an Euclidean space as in Example 6 and let f(x), g(x) R[x]. Cauchy-Bunyakovsky-Schwarz inequality implies ( 1 2 1 1 f(x) g(x) dx) f(x) 2 dx g(x) 2 dx. 0 Equality holds if and only if there exists t R such that either f(x) = t g(x) or g(x) = t f(x). Example 15. Let (R[x],, ) be an Euclidean space as in Example 7 and let f(x), g(x) R[x]. Cauchy-Bunyakovsky-Schwarz inequality implies ( 2 f(x) g(x)e dx) x2 f(x) 2 e x2 dx g(x) 2 e x2 dx. Equality holds if and only if there exists t R such that either f(x) = t g(x) or g(x) = t f(x). Example 16. Let (C([0, 1]),, ) be an Euclidean space as in Example 8 and let f(x), g(x) C([0, 1]). Cauchy-Bunyakovsky-Schwarz inequality implies ( 1 2 1 1 f(x) g(x) dx) f(x) 2 dx g(x) 2 dx. 0 Equality holds if and only if there exists t R such that either f(x) = t g(x) or g(x) = t f(x). Example 17. Let (M n (R),, ) be an Euclidean space as in Example 9 and let A, B M n (R). Cauchy-Bunyakovsky-Schwarz inequality implies ( Tr(AB t ) ) 2 Tr(AA t ) Tr(BB t ). 0 0 0 0 15 Equality holds if and only if there exists t R such that either A = t B or B = t A. Let (V,, ) be an Euclidean space and let v V. The number v := < v, v > is called the length or the norm of v. The map : V R>0 has the following three properties: 1) For every vector v V, v 0. Equality holds if and only if v = O. 2) For any v V, a R, av = a v. 3) For any u, v V, u + v u + v.

The third property is called the triangle inequality. The vector v V is called a unit vector if v = 1. If v V and v O then v/ v is a unit vector. We define vectors u, v V to be orthogonal or perpendicular, and write u v, if their inner product u, v is zero. Vector system {v 1, v 2,..., v n } of V is called orthogonal if its elements are mutually perpendicular, i. e., v i, v j = 0 whenever i j. If in addition each vector of the system has length 1 then the system is called orthonormal. Component and projection. Let u be a nonzero vector of an Euclidean space (V,, ). (Then u > 0.) For any v V there exists a unique number c such that the vector v cu is perpendicular to u. Indeed, we have v cu u v cu, u = 0 c = We call c the component of v along u. projection of v along u. v, u u, u = v, u u. 16 The vector cu is called the Proposition 18. Let {v 1, v 2,..., v n } be an orthogonal system of nonzero vectors of an Euclidean space (V,, ) and let c i be the component of v V along v i. Then the vector is perpendicular to each v i. v c 1 v 1 c 2 v 2 c n v n The next theorem shows that c 1 v 1 + c 2 v 2 + + c n v n gives the closest approximation of v as a linear combination of v 1, v 2,..., v n. Theorem 19. Let {v 1, v 2,..., v n } be an orthogonal system of nonzero vectors of an Euclidean space (V,, ) and let c i be the component of v V along v i. Let a 1, a 2,..., a n be numbers. Then n v n c i v i v a i v i. i=1 Theorem 20 (Bassel inequality). Let {v 1, v 2,..., v n } be a orthonormal system of vectors of an Euclidean space (V,, ) and let c i be the component of v V along v i. Then n c 2 i v 2. i=1 i=1

Gram-Schmidt orthogonalization process. Denote by L(v 1, v 2,..., v n ) the subspace of the vector space V generated by the vector system {v 1, v 2,..., v n } of V. Then L(v 1, v 2,..., v n ) = {α 1 v 1 + α 2 v 2 + + α n v n α 1, α 2,..., α n R}. Theorem 21. Suppose that (V,, ) is an Euclidean space and let {v 1, v 2,..., v n } be a vector system of V which is linearly independent over R. Then there exists an orthonormal vector system {u 1, u 2,..., u n } such that for every j {1, 2,..., n} we have L(v 1, v 2,..., v j ) = L(u 1, u 2,..., u j ). Corollary 22. Let (V,, ) be a finite dimensional Euclidean space. Assume that V O. Then V has an orthogonal basis. The method of finding the orthonormal vector system {u 1, u 2,..., u n } in Theorem 21 is known as Gram-Schmidt orthogonalization process. This procedure can be described explicitly. First we construct an orthogonal vector system {v 1, v 2,..., v n} as follows. v 1 = v 1, 17 v 2 = v 2 v 2, v 1 v 1, v 1 v 1, v 3 = v 3 v 3, v 2 v 2, v 2 v 2 v 3, v 1 v 1, v 1 v 1, (2.1) v n = v n v n, v n 1 v n 1, v n 1 v n 1 v n, v 1 v 1, v 1 v 1. Then the vector system {u 1 := v 1/ v 1, u 2 := v 2/ v 2,..., u n := v n/ v n } is orthonormal and satisfies the statement of Theorem 21. Example 23. Ortogonalize the vector system v 1 = (1, 2, 1), v 2 = ( 3, 4, 1), v 3 = ( 4, 7, 0). Let v 1 := v 1 and v 2 = v 2 v 2, v 1 v 1, v 1 v 1. In other words, we subtract from v 2 its projection along v 1. Hence the vector v 2 is perpendicular to v 1, by Proposition 18. We find v 2 = ( 1, 0, 1).

18 Now we subtract from v 3 its projection along v 1 and v 2: v 3 = v 3 v 3, v 2 v 2, v 2 v 2 v 3, v 1 v 1, v 1 v 1. The vector v 3 is perpendicular to v 1 and v 2, by Proposition 18. We find v 3 = (1, 1, 1). Now the vector system {v 1, v 2, v 3} is orthogonal and satisfies the statement of Theorem 21, i. e., L(v 1 ) = L(v 1), L(v 1, v 2 ) = L(v 1, v 2), L(v 1, v 2, v 3 ) = L(v 1, v 2, v 3). If we wish to have an orthonormal vector system then we divide these vectors by their length: u 1 := v 1 v 1 = 1 6 (1, 2, 1), u 2 := v 2 v 2 = 1 2 ( 1, 0, 1), u 3 := v 3 v 3 = 1 3 (1, 1, 1). Orthogonal complement. Let (V,, ) be an Euclidean space and let W be a subspace of V. Denote by W the set of all vectors of V which are perpendicular to every vector of W, i. e., W = {u V u v for every v W }. Then W is a subspace of V which is called the orthogonal complement of W. Theorem 24. Let W be a subspace of of an Euclidean space (V,, ). Then V is a direct sum of W and the orthogonal complement W. In other words, W W = {O} and dim W + dim W = dim V. (2.2)

Finding the orthogonal complement. Let W be a subspace of of V and let v V. Since V is a direct sum of W and W, there exist unique vectors u W and w W such that v = u + w. The vector u is called the projection of v onto the subspace W. Similarly, the vector w is called the perpendicular of v to the subspace W. Note that the vector w is the projection of v onto the orthogonal complement W. Suppose that W = L(v 1, v 2,..., v m ) is a subspace of V and we want to find the orthogonal complement W. There exist vectors v m+1,..., v n such that {v 1, v 2,..., v n } is a basis of V. Orthogonalising the system {v 1, v 2,..., v n } we obtain an orthonormal vector system {u 1, u 2,..., u n } which satisfies the statement of Theorem 21. Then {u m+1, u m+2,..., u n } is an orthonormal basis of W, i. e., W = L(u m+1, u m+2,..., u n ). Example 25. Consider the usual Euclidean space R 4. Find a basis of the orthogonal complement W of a subspace W generated by vectors v 1 = (1, 4, 5, 2) and v 2 = (2, 7, 1, 3). One could find a basis of W by considering a basis of V which contains vectors v 1 and v 2 and then orthogonalising it. However we will do it in a different way. Indeed, a vector v = (x 1, x 2, x 3, x 4 ) R 4 lies in W if and only if it is perpendicular to both vectors v 1 and v 2 which generate W, i. e., v v v W 1 v, v 1 = 0 v v 2 v, v 2 = 0. Hence the orthogonal complement W coincides with the set of solutions of the system of liear equations 19 x 1 + 4x 2 + 5x 3 2x 4 = 0 2x 1 + 7x 2 + x 3 + 3x 4 = 0. (2.3)

20 We find that x 1 = 31x 3 26x 4 x 2 = 9x 3 + 7x 4. Hence the set of solutions of the system (2.3) is {(31x 3 26x 4, 9x 3 + 7x 4, x 3, x 4 ): x 3, x 4 R}. Substituting x 3 = 1 and x 4 = 0 we obtain the vector u 1 = (31, 9, 1, 0) W, and substituting x 3 = 0 and x 4 = 1 we obtain the vector u 1 = ( 26, 7, 0, 1) W. On the other hand, it follows from (2.2) that dim W = dim R 4 dim W = 4 2 = 2. Hence the system {u 1, u 2 } is a basis of W. Finding the projection. Let W be a subspace of an Euclidean space (V,, ). Suppose that {u 1, u 2,..., u m } is an orthogonal basis of W and {u m+1, u m+2,..., u n } is an orthogonal basis of W. Let v be a vector. Then the projection of v onto the subspace W is α 1 u 1 + α 2 u 2 +..., α m u m, α j = v, u j, j = 1, 2,..., m, u j, u j and the perpendicular of v to the subspace W is α m+1 u m+1 +α m+2 u m+2 +..., α n u n, α j = v, u j, j = m+1, m+2,..., n. u j, u j Example 26. Find the projection and the perpendicular of the vector v = (2, 2, 3, 3) to the subspace W = L((1, 1, 2, 3), ( 1, 3, 1, 5)) of Euclidean space R 4. Denote by u the projection of v onto W. Then u W, and therefore u = x (1, 1, 2, 3) + y ( 1, 3, 1, 5) for some x, y R. Since v u = (2 x + y, 3 + x 3y, 3 2x y, 3 3x 5y) is the perpendicular of v to W, this vector is orthogonal to both vectors (1, 1, 2, 3) and ( 1, 3, 1, 5), i. e., v u, (1, 1, 2, 3) = 0 and v u, ( 1, 3, 1, 5).

21 This implies solving the system of linear equations 15x + 13y = 2 13x + 36y = 23. We find x = 1, y = 1. Therefore the projection of v onto the subspace W is u = (2, 4, 1, 2) and the perpendicular of v to W is (0, 1, 2, 1). Exercises. Exercise 1. Orthogonalise the following vector systems: a) v 1 = (2, 1, 3), v 2 = (5, 3, 5), v 3 = (4, 4, 6) ; b) v 1 = (1, 1, 2, 1), v 2 = (0, 6, 1, 1), v 3 = (7, 10, 5, 0) ; c) v 1 = (2, 2, 1, 4), v 2 = ( 4, 5, 1, 14), v 3 = ( 5, 8, 5, 9); d) v 1 = (1, 3, 4, 2), v 2 = (5, 1, 5, 1), v 3 = ( 5, 13, 5, 3), v 4 = (6, 8, 8, 10). Answer: a) u 1 = (2, 1, 3), u 2 = (1, 1, 1), u 3 = (4, 5, 1) ; b) u 1 = (1, 1, 2, 1), u 2 = (1, 5, 1, 2), u 3 = (4, 1, 1, 5); c) u 1 = (2, 2, 1, 4), u 2 = (2, 1, 2, 2), u 3 = ( 7, 8, 2, 1); d) u 1 = (1, 3, 4, 2), u 2 = (4, 2, 1, 1), u 3 = (1, 3, 1, 3), u 4 = (5, 3, 7, 7). Exercise 2. Append the following vector systems to obtain an orthonormal basis of Euclidean space R n. a) v 1 = 1 3 (1, 1, 5), v 3 2 = 1 14 ( 2, 3, 1) ; b) v 1 = 1(0, 1, 2, 2), v 3 2 = 1 3 (3, 4, 1, 1) ; 3 c) v 1 = 1(5, 3, 1, 1), v 6 2 = 1 (1, 1, 5, 3). 6 Answer: a) u 1 = 1 3 3 (1, 1, 5), u 2 = 1 14 ( 2, 3, 1), u 3 = 1 b) u 1 = 1 3 (0, 1, 2, 2), u 2 = 1 3 3 (3, 4, 1, 1), u 3 = 1 3 ( 16, 11, 1) ; 42 2 (0, 0, 1, 1), u 4 = 1 3 ( 6, 4, 1, 1) ; 6 c) u 1 = 1(5, 3, 1, 1), u 6 2 = 1(1, 1, 5, 3), u 6 3 = 1 ( 3, 5, 1, 1), 6 u 4 = 1 (1, 1, 3, 5). 6

Exercise 3. Consider the Euclidean space defined in Example 8 which consists of all continuous real-valued functions on the interval [0, 1]. Let W be the subspace of functions generated by the two functions f(x) and g(x) such that f(x) = x, g(x) = x 2. Find an orthonormal basis of W. Exercise 4. Consider the Euclidean space defined in Example 8. Let W be the subspace generated by the three functions 1, x and x 2. Find an orthonormal basis of W. Exercise 5. Find a basis of the orthogonal complement W of a subspace W of Euclidean space R 4. a) W = L ((1, 4, 5, 2), (2, 7, 1, 3)) ; b) W = L ((1, 3, 5, 7), (2, 5, 3, 4), (3, 7, 2, 0)) ; c) W = L ((2, 2, 5, 3), (3, 4, 1, 2), (5, 8, 13, 12)). Answer: a) {(31, 9, 1, 0), ( 26, 7, 0, 1)}; b) {( 241, 103, 1, 9)}; c) {(22, 17, 2, 0), ( 16, 13, 0, 2)}. Exercise 6. Consider the Euclidean space defined in Example 9 which consists of all n n matrixes whose coefficients are real numbers. Describe the orthogonal complement of the subspace of diagonal matrices. What is the dimension of this complement? Exercise 7. Find the projection and the perpendicular of a vector v to a subspace W of Euclidean space R 4. a) v = (8, 2, 7, 9), W =< (4, 5, 1, 3), ( 1, 2, 7, 4) > ; b) v = (3, 2, 3, 3), W =< (1, 0, 1, 0), (0, 0, 1, 1), (2, 0, 0, 1) > ; c) v = (2, 0, 1, 1), W =< (1, 1, 0, 0), (0, 1, 1, 0), (0, 0, 1, 4), (0, 0, 0, 1) >. Answer: a) The projection is (3, 7, 6, 7) and the perpendicular is (5, 5, 1, 2) ; b) The projection is (3, 0, 3, 3) and the perpendicular is (0, 2, 0, 0) ; c) The projection is (2, 0, 1, 1) and the perpendicular is (0, 0, 0, 0). 22

Exercise 8. Find a basis of the orthogonal complement W of a subspace 2x 1 3x 2 + 4x 3 4x 4 = 0 W = (x 1, x 2, x 3, x 4 ) : 3x 1 x 2 + 11x 3 13x 4 = 0 4x 1 + x 2 + 18x 3 23x 4 = 0 of Euclidean space R 4. Answer: {(2, 3, 4, 4), (3, 1, 11, 13), (4, 1, 18, 23)}. Exercise 9. Let (V,, ) be a finite dimensional Euclidean space. Let {v 1, v 2,..., v n } be an orthonormal system of vectors in V. Assume that for every v V we have n v, v i 2 = v 2. i=1 Show that {v 1, v 2,..., v n } is a basis of V. Exercise 10. Let (V,, ) be an Euclidean space. Prove the parallelogram law, for any vectors u, v V, u + v 2 + u v 2 = 2 ( u 2 + v 2). 23

3. Linear maps 24 What is linear map? Let (V, +) and (W, +) be two (possibly, distinct) vector spaces over the field k. A map (function) f : V W is said to be linear if f satisfies two important requirements: 1) f(αv) = αf(v) for all v V, α K; 2) f(u + v) = f(u) + f(v) for all vectors u, v V. In other words, linear map preserves the linear operations (the addition of vectors and the multiplication by scalars) in vector spaces. Example 11. Check whether a map f : R 2 R 2, which maps the vector v = (x 1, x 2 ) R 2 into is linear. f(v) = (2x 1 x 2, 3x 1 + x 2 ) Solution. We shall check both properties. Let α R be a scalar. Then αv = (αx 1, αx 2 ). By the definition of f, f(αv) = (2αx 1 αx 2, 3αx 1 + αx 2 ) = α(2x 1 x 2, 3x 1 + x 2 ) = αf(v). Hence the first requirement is met. Now we check the second. Let u = (x 1, x 2 ) and v = (y 1, y 2 ) be two vectors in R 2. Then u+v = (x 1 +y 1, x 2 +y 2 ). Using the definition of f, we get: f(u + v) = (2(x 1 + y 1 ) (x 2 + y 2 ), 3(x 1 + y 1 ) + (x 2 + y 2 )) = = (2x 1 x 2, 3x 1 + x 2 ) + (2y 1 y 2, 3y 1 + y 2 ) = f(u) + f(v). Thus f is a linear map from V = R 2 to W = R 2. Example 12. We consider the set V = C[0, 1] of continuous real functions, defined in the interval [0, 1] as a vector space over R. We define the map f which takes the vector v = v(t) and maps it into the real number f(v) = 1 0 v(t)dt. Check whether this map is a linear map from C[0, 1] to R.

Solution. Let α be a real scalar and let u(t), v(t) be two continuous functions, defined in [0, 1]. By the properties of a definite integral, one has f(αv) = 1 αv(t)dt = α 1 0 0 v(t)dt = αf(v), 25 f(u + v) = 1 (u(t) + v(t))dt = 1 u(t)dt + 1 0 0 0 Hence f is a linear map from V = C[0, 1] to W = R. v(t)dt = f(u) + f(v). Obviously, not every linear map f : V W is linear. Example 13. Check whether the function f(x) = x 2 is a linear map from R to R. Solution. In order to show that the map is non linear, it suffices to prove that at least one of the two properties in the definition of linear map are not satisfied. We will check both. First, let x 0, α 0, 1. Then f(αx) = α 2 x 2 αx 2 = αf(x), Secondly, let x, y be non-zero real numbers. Then f(x + y) = (x + y) 2 = x 2 + 2xy + y 2 x 2 + y 2 = f(x + y). Thus, neither of the two linear properties hold. What is matrix of linear map? Suppose that V and W have finite dimensions, say, dim K V = m, dim K W = n. First, lets pick bases for these vector spaces: let v 1, v 2,..., v m be the basis of V, and let w 1, w 2,..., w n be the basis of W. Let f : V W be a linear map. Express the images of vectors v 1, v 2,..., v m under f in the basis of W : f(v 1 ) f(v 2 ) = α 11 w 1 + α 12 w 2 + + α 1n w n = α 21 w 1 + α 22 w 2 + + α 2n w n f(v m ) = α m1 w 1 + α m2 w 2 + + α mn w n.

Now, copy the coordinates α ij into the table if size m n. The matrix we obtain this way is called the matrix of the linear map f (in respective bases of V and W ): α 11 α 12 α 1n α A = 21 α 12 α 1n... α m1 α m2 α mn Given the matrix of f, one can find the image f(v) of any vector v V. Suppose that v = β 1 v 1 + β 2 v 2 +... β m v m, so that the coordinates of v in the basis v 1, v 2,..., v m are Then the coordinates of the image v = (β 1, β 2,..., β m ). f(v) = γ 1 w 1 + γ 2 w 2 + + γ n w n, 26 or, f(v) = (γ 1, γ 2,..., γ n ) in the basis w 1, w 2,..., w n can be computed easily using matrix multiplication: α 11 α 12 α 1n α f(v) = va = (β 1, β 2,..., β m ) 21 α 12 α 1n... = (γ 1, γ 2,..., γ n ). α m1 α m2 α mn Example 14. A linear map f maps the vectors v 1, v 2, v 3, which form the basis of three dimentional space V into two dimentional vector space W : f(v 1 ) = w 1 + 4w 2, f(v 2 ) = 5w 1 + 3w 2, f(v 3 ) = 2w 1 5w 2,

where w 1 and w 2 are the basis of W. Write the matrix of the linear map f and find the image f(v) of a vector v = 7v 1 2v 2 v 3. Solution. We copy the coordinates of the images f(v 1 ), f(v 2 ), f(v 3 ) in W into 3 2 matrix 1 4 A = 5 3 2 5 The coordinates of v V in basis v 1, v 2, v 3 are v = (7, 2, 1). Hence, 1 4 f(v) = va = (7, 2, 1) 5 3 = ( 19, 27). 2 5 27 Thus f(v) = 19w 1 + 27w 2. Remark 15. If vector spaces V and W coincide (V = W, m = n), then the linear map f : V V is called a linear transformation of V. The matrix of a linear transformation (in basis v 1 = w 1, v 2 = w 2,..., v n = w n ) is called transformation matrix. Transformation matrix is a square matrix of the size n n. Example 16. Write the transformation matrix of the linear transformation f : R 3 R 3 f(x 1, x 2, x 3 ) = (x 1 x 3, 2x 2 + x 3, 4x 1 x 2 ) in the standard basis e 1, e 2, e 3 and find the image of the vector v = (1, 1, 1). Solution. The standard basis of R 3 is e 1 = (1, 0, 0), e 2 = (0, 1, 0), e 3 = (0, 0, 1). Thus f(e 1 ) = (1, 0, 4) = e 1 + 0e 2 + 4e 3 ; f(e 2 ) = (0, 2, 1) = 0e 1 + 2e 2 e 3 ; f(e 3 ) = ( 1, 1, 0) = e 1 + e 2 + 0e 3.

28 Hence, the transformation matrix is 1 0 4 A = 0 2 1. 1 1 0 If we plug in the coordinates of v into f, we get f(1, 1, 1) = (0, 3, 3). We could get the same result by computing f(v) = va. Example 17. Find the matrix of the linear map f : V V, which maps the vectors u 1 = (1, 2, 3), u 2 = ( 1, 1, 5), u 3 = (2, 5, 5) into vectors v 1 = (1, 0, 1), v 2 = (0, 1, 1), v 3 = ( 1, 1, 0), respectively (the coordinates of the vectors are given in some unknown basis of V ). Solution. In this example, we do not know the V basis. However, we do not need it, since the coordinates are given. Since dim K V = 3, the size of the transformation matrix should be 3 3. Let α 11 α 12 α 13 A = α 21 α 22 α 23, α 31 α 32 α 33 where α i,j are unknown coefficients. After multiplying vectors u 1, u 2, u 3 by the j th column of the matrix A, we will obtain j th coordinates of vectors v 1, v 2, v 3. Hence we obtain three different systems of linear equations: α 11 +2α 21 +3α 31 = 1 α 12 +2α 22 +3α 32 = 0 α 11 α 21 5α 31 = 0, α 12 α 22 +5α 32 = 1, 2α 11 +5α 21 +5α 31 = 1 2α 12 +5α 22 +5α 32 = 1 α 13 +2α 23 +3α 33 = 1 α 13 α 23 5α 33 = 1. 2α 13 +5α 23 +5α 33 = 0 The solution of these three systems would be a heroic endeavor. However, one can make one important simplification. Let us observe that the coefficients on the left side are the same in all three systems. Only the constants

on right side are different. We will use this fact to our advantage by solving all three systems simultaneously. Let us write all the coefficients into one matrix: 1 2 3 1 0 1 1 1 5 0 1 1. 2 5 5 1 1 0 The rows in the left side of the matrix are simply the vectors u 1, u 2, u 3 ; rows in the right side are vectors v 1, v 2, v 3. We will do the Gaussian elimination on rows to reduce the left side into the identity matrix: 1 2 3 1 0 1 1 ( 2) 1 1 5 0 1 1 2 5 5 1 1 0 1 2 3 1 0 1 0 1 2 1 1 2 0 0 1 4 0 4 2 1 0 0 27 2 25 0 1 0 7 1 6. 0 0 1 4 0 4 ( 3) ( 1) ( 2) The solutions to the linear systems of equations are located in the right hand side of the above matrix. Hence the transformation matrix is: 27 2 25 A = 7 1 6. 4 0 4 Remark 18. The method we applied in the solution can be used to solve more general matrix equations AX = B, where A, B - are two given matrices, and X is an unknown matrix. We have already used it to compute the inverse matrices. 29

Change of basis. Let f : V W be a linear map. Suppose that a new basis v 1, v 2,..., v m have been picked in the vector space V ; while another new basis w 1, w 2,..., w n have been chosen for the space W. Then the new matrix A of the linear mat f can be computed as follows: A = T AR 1, where A is a matrix of f in old bases, T is a transition matrix from the basis v 1, v 2,..., v m to the basis v 1, v 2,..., v m in space V, and R is a transition matrix from the basis w 1, w 2,..., w n to the basis w 1, w 2,..., w n in space W. As a separate case of this formula, for a linear transformation f (i.e. V = W, and the bases coincide), one has T = R and the formula simplifies to: A = T AT 1 The matrix T AT 1 is called conjugate to A. Example 19. The matrix A is the matrix of a linear transformation f : V V in basis v 1, v 2, v 3 : 1 0 2 A = 0 3 2. 1 1 0 Find the transformation matrix in the new basis u 1 = 2v 1 +v 2, u 2 = v 1 +v 2, u 3 = v 3. Solution. The transition matrix T from basis v 1, v 2, v 3 to basis u 1, u 2, u 3 is 2 1 0 1 1 0 T = 1 1 0, T 1 = 1 2 0. 0 0 1 0 0 1 By the conjugate matrix formula, the transformation matrix in the new basis u 1, u 2, u 3 : 2 1 0 1 0 2 1 1 0 5 8 2 A = T AT 1 = 1 1 0 0 3 2 1 2 0 = 4 7 0. 0 0 1 1 1 0 0 0 1 0 1 0 30

31 Important properties of matrices. Let f and g be linear maps from V to W with respective matrices A ir B (in some bases of V and W ). Then for any scalar α k, the matrices of linear maps αf and f + g are αa and A + B, respectively. Next, let us suppose that linear maps f : U V, g : V W have matrices A and B. Then the matrix of a composite map g(f(v)) is equal to the product AB of matrices A and B. The identity matrix 1l n is a matrix of the identity transformation f(v) = v. A linear transformation f : V V is said to be non degenerate if the inverse map f 1 exists. The transformation f is invertible if and only if the matrix A is non degenerate, det A 0. In such case, the inverse matrix A 1 is the matrix of the inverse transformation f 1. Example 20. Linear transformations f and g in space R 2 have matrices A and B, respectively: A = 2 1, B = 1 1. 5 3 0 1 Find whether a linear transformation h(v) = g(f 1 (v)) 3g(v) + 2v exists and compute the matrix C of the transformation h. Solution. Since det A = 1, we know that f is non degenerate and inverse transformation f 1 exists. In the formula, we replace the transformations with matrices (v = 1l 2 v) and find C = A 1 B 3B + 21l = 3 1 1 1 3 1 1 + 2 1 0 5 2 0 1 0 1 0 1 Hence C = 2 5 1. 4

32 Exercises. Exercise 1. Check whether a given map f : V W is linear: a) V = W = R 2, f(x 1, x 2 ) = (x 1 x 2, 3x 1 + 4x 2 ); b) V = W = R 2, f(x 1, x 2 ) = (x 2, x 1 + 5); c) V = C[0, 1], W = R, f(v) = 1 0 v2 (t)dt; d) V = C[0, 1], W = R, f(v) = 1 0 v(t3 ) cos tdt; e) V = W = R[x],f(v) = (x 3 + 1) v; f) V = W = R[x], f(v) = x v. Answer: a) Yes; b) No; c) No; d) Yes; e) Yes; f) Yes. Exercise 2. Find the matrices of linear maps f : V W in given bases; compute the image f(v) of a given vector v V : a) f(v 1 ) = w 1 + w 2 w 3, f(v 2 ) = w 1 w 2 w 3, f(v 3 ) = w 1 + w 3, where v 1, v 2, v 3 are the basis of V and w 1, w 2, w 3 are the basis of W ; vector v = v 1 v 2 v 3 ; b) V = R 3, W = R 2, f(x 1, x 2, x 3 ) = (x 1 + x 2 x 3, x 1 x 2 x 3 ) in standard bases; v = (2, 0, 1); c) V = R 2, W = R 3, f(x 1, x 2 ) = (4x 1 + 5x 2, 3x 1 2x 2, x 1 + x 2 ), V has basis v 1 = ( 1, 1), v 2 = (2, 1), W has basis e 1, e 2 e 3 ; v = (1, 1); d) V = R 4 [x], W = R 3 [x]; f(v) = v (v - denotes a derivative of a polynomial v) in standard basis 1, x,..., x n of R n [x]; v = 1 + x + x 2 + x 4 ; e) V = R 2 [x], W = R 3 [x]; f(v) = x v(t)dt in standard bases; v = 0 1 + 2x + x 2 ; f) V = W = M 2 [R], f(v) = Bv, in standard basis e 1, e 2, e 3, e 4, where B = 1 3, 1 7 e 1 = 1 0, e 2 = 0 1, e 3 = 0 0, 0 0 0 0 1 0 e 4 = 0 0, v = 2 4 0 1 3 1 Answer:

1 1 1 a) A = 1 1 1, f(v) = w 1 + w 3 ; 1 0 1 1 1 b) A = 1 1, f(v) = (1, 1); 1 1 c) A = 4 3 1, f(v) = (9, 1, 2); 5 2 1 0 0 0 0 1 0 0 0 d) A = 0 2 0 0, f(v) = 1 + 2x + 4x 3 ; 0 0 3 0 0 0 0 4 0 1 0 0 e) A = 0 0 1/2 0, f(v) = x + x2 + x 3 /3; 0 0 0 1/3 1 0 1 0 0 1 0 1 f) A =, f(v) = 11 7. 3 0 7 0 23 11 0 3 0 7 33 Exercise 3. Find the matrix of a linear transformation f : V V, which maps vectors v 1, v 2,..., v n into vectors f(v 1 ), f(v 2 ),..., f(v n ): a) v 1 = (1, 3), v 2 = ( 2, 5), f(v 1 ) = ( 2, 1), f(v 2 ) = (2, 1); b) v 1 = (1, 0, 1), v 2 = ( 1, 1, 0), v 3 = (1, 1, 1), f(v 1 ) = ( 2, 1, 0), f(v 2 ) = (0, 1, 1), f(v 3 ) = ( 1, 1, 1); c) v 1 = (1, 1, 2), v 2 = (1, 2, 3), v 3 = (1, 2, 4), f(v 1 ) = (3, 1, 0), f(v 2 ) = (2, 4, 1), f(v 3 ) = (1, 2, 0);

d) v 1 = (1, 2, 1, 1), v 2 = (2, 3, 2, 3), v 3 = (1, 2, 0, 2), v 4 = (1, 3, 2, 0), f(v 1 ) = (1, 1, 0, 2), f(v 2 ) = (0, 2, 0, 1), f(v 3 ) = (1, 0, 1, 1), f(v 4 ) = (2, 0, 0, 1); 34 Answer: a) A = 4 2 1 1 0 ; b) A = 1 0 1 2 1 ; 3 0 0 c) A = 1 8 3 19 5 0 0 0 11 2 ; d) A = 1 2 1 2. 1 1 0 6 1 6 1 1 2 1 7 Exercise 4. Given the matrix A of a linear map f : V W in old basis, find the matrix of f in new basis: a) V = W = R 2, A = 3 1 in old basis v 1, v 2 ; new basis u 1 = 0 1 3v 1 + 4v 2, u 2 = 2v 1 + 3v 2 ; b) V = W = R 2, A = 2 0 old basis v 1, v 2 ; new basis u 1 = 1 3 v 1 + 3v 2, u 2 = 3v 1 + 8v 2 ; 1 0 1 c) V = W = R 3, A = 0 1 0 old basis v 1, v 2, v 3 ; new basis 1 1 0 u 1 = v 1, u 2 = v 2 5v 3, u 3 = 4v 1 + v 2 6v 3 ; 1 1 1 d) V = W = R 3, A = 1 1 1 old basis v 1, v 2, v 3 ; new basis 1 1 1 u 1 = v 1 3v 2 + v 3, u 2 = 2v 1 + 7v 2, u 3 = v 1 + v 2 + 8v 3 ;

Answer: 1 1 e) V = R 3, W = R 2 A = 1 1 old bases are standard bases; new 0 1 basis for V is v 1 = (1, 4, 0), v 2 = ( 5, 19, 0), v 3 = ( 1, 1, 1), new basis for W is w 1 = (3, 2), w 2 = (1, 1); f) V = R 2, W = R 3 A = 2 0 3 old bases are standard bases; 1 0 0 new basis for V is v 1 = (5, 2), v 2 = (2, 1), new basis for W is w 1 = (1, 0, 6), w 2 = (0, 1, 1), w 3 = (2, 0, 11). a) b) c) T 1 = 3 4 ; T AT 1 = 25 33 ; 2 3 16 21 T 1 = 8 3 ; T AT 1 = 35 12 ; 3 1 88 30 1 0 0 5 1 1 T 1 = 20 6 5 ; T AT 1 = 125 36 30 ; 4 1 1 166 46 39 d) 56 25 7 109 48 14 T 1 = 16 7 2 ; T AT 1 = 379 168 48 ; 9 4 1 486 218 60 35 e) R 1 = 1 2 2 1 ; T AR 1 = 10 6 1 3 ; 1 3

36 f) 11 0 6 R 1 = 2 1 1 ; T 162 0 87 AR 1 = ; 67 0 36 2 0 1 Exercise 5. Linear maps f : R 2 R 2 g : R 2 R 2 h : R 3 R 2 have matrices A, B, C in standard bases of R 2 and R 3 : A = 1 2, B = 2 1 0 4, C = 3 2 1 1 4 2. 1 0 Find which of the linear maps, given bellow, exist, and compute their matrices in standard bases: a) f(v) + g 1 (v), v R 2 ; b) g(f(v)) v, v R 2 ; c) (f + g) 1 (v) d) f 1 (g(v) v), v R 2 ; e) h(g(v)) + 2v, v R 2 ; f) h 1 (f(v))), v R 2 ; g) f 1 (h(v)), v R 3. Answer: a) No, since g 1 does not exist b) Yes, AB 1l 2 = 5 3 ; 2 2 1/14 3/14 c) Yes, (A + B) 1 = ; 5/14 1/14 d) Yes, (B 1l 2 )A 1 = 4 7 ; 7 11

e) No, since the image of R 2, g(r 2 ) does not lie in the domain of definition of h, which is R 3 matrices C ir B cannot be multiplied because their sizes are not consistent; f) No, since the inverse of h 1 does not exist (the matrix C is not a square matrix); 4 4 g) Yes, CA 1 = 1 4. 1 2 37

4. Eigenvalues and eigenvectors 38 What is eigenvalue and eigenvector? Let (V, +) be a vector space over the field k. Suppose that f : V V is a linear transformation of V. An element λ k is called an eigenvalue of the transformation f, if there exists a non zero vector v V, v O, such that the image f(v) is parallel to v with a scale factor λ: f(v) = λv. The vector v O with this property is called an eigenvector of the linear transformation f which corresponds to the eigenvalue λ. Example 6. Let V = R, f(v) = 3v for all v R. Then every vector v R is an eigenvector which corresponds to an eigenvalue λ = 3. Example 7. Let V = R 2, f(v) = v for all v R 2. Every vector v R 2 is an eigenvector of the transformation f with a corresponding eigenvalue λ = 1. The transformation matrix of f is A = 1l 2 = 1 0 0 1 (in any basis of V ). Hence f(v) = v1l 2 = v. Example 8. Let V = R 2. Suppose that the transformation matrix of f is A = 2 0, 0 1 in basis v 1, v 2 V. Then v 1 is an eigenvector corresponding to an eigenvalue λ = 2, while v 2 is the eigenvector corresponding to the eigenvalue λ = 1. To see this, recall that the coordinates of v 1 in the basis v 1, v 2 are v 1 = (1, 0). Hence f(v 1 ) = v 1 A = (1, 0) 2 0 = (2, 0) = 2v 1. 0 1

Similarly, the coordinates of the vector v 2 in basis v 1, v 2 are v 2 = (0, 1), thus f(v 2 ) = v 2 A = (0, 1) 2 0 = (0, 1) = v 2. 0 1 Example 9. Let V = R 2,and suppose that A = 3 2. 1 0 is a transformation matrix of f in standard basis e 1, e 2. We will check that the vector v 1 = (1, 2) is the eigenvector which corresponds to the eigenvalue λ = 1, and v 2 = (1, 1) is the eigenvector corresponding to an eigenvalue λ = 2. f(v 1 ) = v 1 A = 1 v 1 f(v 2 ) = v 2 A = 2 v 2. How to find eigenvalues? Let us suppose that V is finite dimensional vector space, dim k V = n. The first step is to pick a basis for V and write the transformation matrix for a linear transformation f : V V 39 in this basis: α 11 α 12 α 1n α A = 21 α 22 α 1n.... α n1 α n2 α nn (How to write the matrix of linear transformation, check the chapter on Linear maps from previous lecture). The second step is to expand the determinant α 11 t α 12 α 1n α φ A (t) = det (A t1l n ) = 21 α 22 t α 1n.... α n1 α n2 α nn t

in powers of t. The polynomial we get this way is called a characteristic polynomial of the linear transformation: φ A (t) = a 0 a 1 t + a 2 t 2 + + ( 1) n 1 a n 1 t n 1 + ( 1) n t n. Third step is to compute the roots of the polynomial φ A (t), λ 1, λ 2,..., λ n (multiple roots are repeated according to their multiplicity): φ A (t) = (λ 1 t)(λ 2 t)... (λ n t). These roots are the eigenvalues of the transformation f. Remark 10. The characteristic polynomial φ A (t) and eigenvalues λ 1,..., λ n of the linear transformation are invariant under the change of basis. This means that they do not change if we pick another basis for V. How to find eigenvectors? Suppose that λ K is an eigenvalue of the linear transformation f. Let v = (β 1, β 2,..., β n ) be the coordinates of an eigenvector v which corresponds to the eigenvalue λ. Since f(v) = va and f(v) = λv, we obtain va = λv. Write this equation as v (A λ1l n ) = O. The eigenvectors v lie in the kernel of the matrix B = A λ1l n : α 11 λ α 12 α 1n α (β 1, β 2,..., β n ) 21 α 22 λ α 1n... = (0, 0,..., 0). α n1 α n2 α nn λ To find them, we solve the coordinates β 1,..., β n in the linear system of equations (α 11 λ)β 1 + α 21 β 2 +... + α n1 β n = 0 40 α 12 β 1 + (α 22 λ)β 2 +... + α n2 β n = 0....... α 1n β 1 + α 2n β 2 +... + (α nn λ)β n = 0

The number of linearly independent solutions (eigenvectors) is equal to the defect of the matrix B. We repeat this process to find linearly independent eigenvectors for all different eigenvalues λ of the linear transformation f. Example 11. Find the eigenvalues and corresponding eigenvectors of a linear transformation of R 2 in standard basis given the transformation matrix A = 10 12. 9 11 Solution. We compute the characteristic polynomials of a transformation: 10 t 12 φ A (t) = 9 11 t = = (10 t) ( 11 t) ( 12) 9 = = t 2 + t 2. We factor the characteristic polynomial and find roots : φ A (t) = t 2 + t 2 = (t 1)(t + 2). The eigenvalues of the linear transformation are λ 1 = 1, λ 2 = 2. Now we calculate eigenvectors. Let v = (β 1, β 2 ) be the eigenvector corresponding to the eigenvalue λ 1 = 1. We write the matrix equation (β 1, β 2 ) 10 1 12 = (0, 0), 9 11 1 or, (β 1, β 2 ) 9 12 = (0, 0), 9 12 After calculating the product of matrices, we get the linear system of equations 9β 1 + 9β 2 = 0 12β 1 12β 2 = 0. The solutions are β 1 = t, β 2 = t, t R. We choose t = 1 and obtain the vector v 1 = (1, 1). 41

Next, let v 2 = (β 1, β 2 ) be the eigenvector corresponding to the eigenvalue λ 2 = 2. We write equation 10 ( 2) (β 1, β 2 ) 12 = (0, 0), 9 11 ( 2) or We get the system (β 1, β 2 ) 12 12 = (0, 0). 9 9 12β 1 + 9β 2 = 0 12β 1 9β 2 = 0. The solutions are β 1 = 3t, β 2 = 4t, t R. If we choose t = 1, we get v 2 = (3, 4). Answer: λ 1 = 1, λ 2 = 2 42 v 1 = (1, 1), v 2 = (3, 4). Example 12. Find the eigenvalues and corresponding eigenvectors of a linear transformation 3 0 4 A = 2 1 2 2 0 3 in standard basis of R 3. Solution. We compute the characteristic polynomials of a matrix (the quick way to do that is to expand the determinant by second column): φ A (t) = det(a t1l 3 ) = 3 t 0 4 = 2 1 t 2 = 2 0 3 t

43 3 t 4 = (1 t) 2 3 t = (1 t)(1 t2 ). The characteristic polynomial is φ A (t) = (1 t) 2 (1 + t), Eigenvalues: λ 1 = λ 2 = 1, λ 3 = 1. The eigenvalue 1 has multiplicity 2. Lets find the corresponding eigenvectors v = (β 1, β 2, β 3 ). The matrix equation is 4 0 4 (β 1, β 2, β 3 ) 2 0 2 = (0, 0, 0) 2 0 2 The rank of a matrix 4 0 4 B = A 1l 3 = 2 0 2 2 0 2 is rg (B) = 1, the defect def (B) = 2. Hence the kernel of B is generated by two linearly independent vectors. From the matrix equation, we get linear system of equations 4β 1 + 2β 2 + 2β 3 = 0 0 = 0 4β 1 + 2β 2 + 2β 3 = 0 which is equivalent to equation 2β 1 + β 2 + β 3 = 0. The solutions are (β 1, β 2, β 3 ) = (t + s, 2t, 2s) = t (1, 2, 0) + s (1, 0, 2), t R, s R. We take two linearly independent solutions v 1 = (1, 2, 0), v 2 = (0, 1, 2) which span the kernel of B. These are the eigenvectors corresponding to the eigenvalue λ 1 = λ 2 = 1.

Next we find the eigenvectors corresponding to the eigenvalue λ 3 = 1. Let v = (β 1, β 2, β 3 ). The matrix equation is 2 0 4 (β 1, β 2, β 3 ) 2 2 2 = (0, 0, 0) 2 0 4 The matrix 2 0 4 B = A + 1l 3 = 2 2 2 2 0 4 has rank 2 and defect 1. Hence the solutions to the system of equations 2β 1 + 2β 2 + 2β 3 = 0 2β 2 = 0 4β 1 + 2β 2 + 4β 3 = 0 (β 1, β 2, β 3 ) = (t, 0, t) = t (1, 0, 1), t R are generated by one linearly independent vector. If we choose t = 1, we get v 3 = (1, 0, 1). Answer: Exercises. λ 1 = λ 2 = 1, λ 3 = 1, v 1 = (1, 2, 0), v 2 = (1, 0, 2), v 3 = (1, 0, 1). Exercise 1. Find the eigenvalues and corresponding eigenvectors of the linear transformations in standard basis of R n : a) A = 5 2, b) A = 8 9, c) A = 0 1, 3 0 4 4 1 0 3 5 5 3 2 2 2 0 1 d) A = 4 5 4, e) A = 0 1 0, f) A = 1 0 1, 4 3 2 4 2 3 0 1 1 44

Answers: 1 3 5 0 0 6 2 1 0 2 1 0 0 4 1 0 g) A =, h) A =, 0 0 2 0 0 1 2 0 1 2 0 1 1 6 4 0 a) λ 1 = 2, λ 2 = 3, v 1 = ( 1, 1), v 2 = ( 3, 2). b) λ 1 = λ 2 = 2, v 1 = ( 2, 3). c) λ 1 = i, λ 2 = i, v 1 = (i, 1), v 2 = (1, i). d) λ 1 = 1, λ 2 = λ 3 = 1, v 1 = (2, 1, 1), v 2 = (0, 1, 0), v 3 = ( 1, 0, 1) e) λ 1 = 1, λ 2 = λ 3 = 1, v 1 = (2, 1, 1), v 2 = (0, 1, 0), v 3 = (1, 0, 1) f) λ 1 = λ 2 = λ 3 = 1, v 1 = (1, 1, 1). g) λ 1 = λ 2 = 1, λ 3 = λ 4 = 2, v 1 = (1, 1, 2, 0), v 3 = (0, 0, 1, 0) h) λ 1 = i, λ 2 = i, λ 3 = λ 4 = 3, v 1 = (1, 1 + 2i, i, i), v 2 = (1, 1 2i, i, i), v 3 = (0, 1, 1, 0) 45