Unit 4. Matrices, Linear Maps and change of basis

Size: px

Start display at page:

Download "Unit 4. Matrices, Linear Maps and change of basis"

Justin Randall
5 years ago
Views:

1 Unit 4. Matrices, Linear Maps and change of basis Linear Algebra and Op:miza:on Msc Bioinforma:cs for Health Sciences Eduardo Eyras Pompeu Fabra University hlp://comprna.upf.edu/courses/master_mat/

2 Linear maps are opera:ons on vectors f A : R n R m u! v = f A (u) = Au R m which can be represented as matrix opera:ons: v = Au = a a 2 a 3 a 2 a 22 a 23 a 3 a 32 a 33 u u 2 u 3 = v v 2 v 3

3 For example, a scaling transforma:on in R 2 is a linear map of R 2 onto itself, and can be represented by a 2x2 diagonal matrix: Scaling dila:on (a >), contrac:on (a <) v = Au = a a u u 2 = au au 2 P P = 4 2

4 A reflec:on can be represented by a 2x2 diagonal matrix: v = Au = u u 2 = u u 2 With the special property: A 2 = Reflection P P 2 2 = 2 2

5 Pure rota:ons are represented by orthonormal matrices: A general orthonormal matrix A fulfills: a b A = c d A = A T è a 2 + c 2 = b 2 + d 2 = ab + cd = Rotation u ϑ v = Au Which we can parameterize with a single variable that we can interpret as the angle of rota:on: v = A(θ)u = = cosθ sinθ u cosθ u 2 sinθ u sinθ + u 2 cosθ ( cosθ) 2 + ( sinθ) 2 = sinθ cosθ sinθ cosθ sinθ cosθ = u u 2 ϑ = π 2 u' = ϑ = π 2 u' = A(π / 2)u = u = = 2

6 Defini&on Given 2 vector spaces M, N, a linear map is a map between them: f : M N u! f (u) N Such that ) 2) u, v M f (u + v) = f (u)+ f (v) N λ R,u M f (λu) = λ f (u) N

7 Example of a linear map f : R 2 R (x, y)! f (x, y) We show property (): f (x, y)+ (w, z) ( ) = 2x y ( ) = f ((x + w, y + z) ) = 2(x + w) (y + z) ( ) + f ((w, z) ) = 2x y + 2w z = f (x, y) associa:vity in R We show property (2): ( ) = f ((λx, λy) ) = 2λx λy = λ(2x y) = λ f ((x, y) ) f λ(x, y) defini:on of f defini:on of f associa:vity in R

8 Example f : R 2 R 2 (x, y)! f ((x, y) ) = (x +, 2y) No, because: ( ) = ( x + u +, 2(y + v) ) (x +, 2y)+ (u +, 2v) = f ((x, y) ) + f ((u, v) ) ( ) = f ((λx, λy) ) = (λx +, 2λy) λ(x +, 2y) = λ f ((x, y) ) f (x, y)+ (u, v) f λ(x, y) Linear map? Example f : E k { } u! f u ( ) = k Linear map? No, because: ( ) = k f (u)+ f (v) = 2k ( ) = k λ f (u) = λk f u + v f λu

9 Image of a linear map M N f : M N u! f (u) N Im( f ) Im( f ) = { w N / u M : f (u) = w} N It is all the set of elements from the target space described by the linear map range( f ) = Im( f )

10 The image of a linear map is a vector subspace U V Vector subspace is a subset of a vector space That is closed under the same opera:ons of the vector space: ) U is a vector space under the same vector addi:on 2) It is a vector space under the scalar mul:plica:on 3) It contains the neutral element: u, v U u + v U a R, v U av U U

11 The image of a linear map is a vector subspace f : M N u! f (u) N Im( f ) = { w N / u M : f (u) = w} N Proof f is a linear map ) f (u), f (v) Im( f ) f (u)+ f (v) = f (u + v) Im( f ) (since u+v is a vector in M) 2) a R, f (u) Im( f ) af (u) = f (au) Im( f ) (since u+v is a vector in M) 3) f is a linear map f (u) Im( f ) f () = f (u u) = f (u) f (u) Im( f ) f () Im( f ) f is a linear map

12 Kernel (nullspace) of a linear map Ker( f ) M f : M N u! f (u) N N Null vector Ker( f ) = f () = { v M / f (v) = } M Set of all elements of M that map to the neutral element null( f ) = Ker( f )

13 The Kernel of a linear map is a vector subspace f is a linear map u, v Ker( f ) ) u, v Ker( f ) f (u + v) = f (u)+ f (v) = + = u + v Ker( f ) f is a linear map 2) a R, u Ker( f ) f (au) = af (u) = a au Ker( f ) 3) f () = f (u u) = f (u) f (u) = = Ker( f ) f is a linear map u Ker( f )

14 Example f : R R 2 x! f (x) = (x, ) R 2 ) Is f a linear map? 2) What is Im(f)? 3) What is Ker(f)?

15 Example f : R R 2 x! f (x) = (x, ) R 2 ) Is f a linear map? x, y R f (x + y) = (x + y, ) = (x, )+ (y, ) = f (x)+ f (y) λ R, x R f (λx) = (λx, ) = λ(x, ) = λ f (x) 2) What is Im(f)? 3) What is Ker(f)?

16 Example f : R R 2 x! f (x) = (x, ) R 2 ) Is f a linear map? x, y R f (x + y) = (x + y, ) = (x, )+ (y, ) = f (x)+ f (y) λ R, x R f (λx) = (λx, ) = λ(x, ) = λ f (x) 2) What is Im(f)? Im( f ) = {(x, ), x R} It is a space completely equivalent to R 3) What is Ker(f)?

17 Example f : R R 2 x! f (x) = (x, ) R 2 ) Is f a linear map? x, y R f (x + y) = (x + y, ) = (x, )+ (y, ) = f (x)+ f (y) λ R, x R f (λx) = (λx, ) = λ(x, ) = λ f (x) 2) What is Im(f)? Im( f ) = {(x, ), x R} It is a space completely equivalent to R 3) What is Ker(f)? Ker( f ) = { x R / f (x) = (, ) } Ker( f ) = { } single element

18 Properties of Linear Maps Monomorphism ( injec&ve / one-to-one linear map) Ker( f ) = { } M f : M N u! f (u) N N Null vector One-to-one if f (u) = f (v) u = v Not two elements are mapped to the same element in the image space In par:cular f () = Ker( f ) = { }

19 Properties of Linear Maps Epimorphism ( surjec&ve / onto linear map) M f : M N u! f (u) N N Im( f ) = N Null vector Onto: w N u M / f (u) = w The image covers the en:re target vector space

20 Properties of Linear Maps Isomorphism ( bijec&ve / One-to-one & Onto linear map) Ker( f ) = { } M f : M N u! f (u) N N Im( f ) = N Null vector bijec&ve: Onto: One-to-one w N u M / f (u) = w if f (u) = f (v) u = v Isoforms are linear maps that are both one-to-one and onto

21 Example Consider the vector space of polynomials of degree 2 { } P 2 (R) = v = ax 2 + bx + c / a, b, c R Consider the map between this vector space and R 3 f : P 2 (R) R 3 v = ax2 + bx + c! f (v) = (c, b, a) R 3 Is f a linear map? Is it one-to-one (injec:ve)? Is it Onto (surjec:ve)?

22 Example Consider the vector space of polynomials of degree 2 { } P 2 (R) = v = ax 2 + bx + c / a, b, c R Consider the map between this vector space and R 3 f : P 2 (R) R 3 v = ax2 + bx + c! f (v) = (c, b, a) R 3 Is f a linear map? u,u 2 P 2 (R), u = a x 2 + b x + c, u 2 = a 2 x 2 + b 2 x + c 2 f (u + v) = (a + a 2, b + b 2,c + c 2 ) = (a, b,c )+ (a 2, b 2, c 2 ) = f (u)+ f (v) ) 2) λ R,u P 2 (R) f (λu) = (λa, λb, λc) = λ(a, b, c) = λ f (u) Is it one-to-one (injec:ve)? Is it Onto (surjec:ve)?

23 Example Consider the vector space of polynomials of degree 2 { } P 2 (R) = v = ax 2 + bx + c / a, b, c R Consider the map between this vector space and R 3 f : P 2 (R) R 3 v = ax2 + bx + c! f (v) = (c, b, a) R 3 Is f a linear map? u,u 2 P 2 (R), u = a x 2 + b x + c, u 2 = a 2 x 2 + b 2 x + c 2 f (u + v) = (a + a 2, b + b 2,c + c 2 ) = (a, b,c )+ (a 2, b 2, c 2 ) = f (u)+ f (v) ) 2) λ R,u P 2 (R) f (λu) = (λa, λb, λc) = λ(a, b, c) = λ f (u) Is it one-to-one (injec:ve)? Is it Onto (surjec:ve)? f (u ) = f (u 2 ) (a, b, c ) = (a 2, b 2, c 2 ) u = u 2

24 Example Consider the vector space of polynomials of degree 2 { } P 2 (R) = v = ax 2 + bx + c / a, b, c R Consider the map between this vector space and R 3 f : P 2 (R) R 3 v = ax2 + bx + c! f (v) = (c, b, a) R 3 Is f a linear map? u,u 2 P 2 (R), u = a x 2 + b x + c, u 2 = a 2 x 2 + b 2 x + c 2 f (u + v) = (a + a 2, b + b 2,c + c 2 ) = (a, b,c )+ (a 2, b 2, c 2 ) = f (u)+ f (v) ) 2) λ R,u P 2 (R) f (λu) = (λa, λb, λc) = λ(a, b, c) = λ f (u) Is it one-to-one (injec:ve)? Is it Onto (surjec:ve)? f (u ) = f (u 2 ) (a, b, c ) = (a 2, b 2, c 2 ) u = u 2 (a, b, c) R 3 u = ax 2 + bx + c P 2 (R) v R 3, u P 2 (R) / f (u) = v

25 Properties of Linear Maps If we find an isomorphism (linear map + onto + one-to-one) between two vector spaces, we say that both vector spaces are isomorphic In general P 2 (R) R 3 P N (R) R N+ Two isomorphic vector spaces have the same dimension: M N dim(m ) = dim(n)

26 Exercise: Consider the vector space of func:ons on a variable θ Show that it is isomorphic to R 2 G(ϑ ) = { acosϑ + bsinϑ / a, b R} G(ϑ ) R 2 Hint: build a map between both vector spaces f : G(ϑ ) R 2 g = acosϑ + bsinϑ! f (g) = (a, b) R 2 Show that this map is linear, one-to-one and onto

27 Consider a linear map from R 2 to itself With the canonical basis of R 2 e = f : R 2 R 2 u! f (u), e 2 = We define f by the opera:on on the basis vectors: f (e ) = e, f (e 2 ) = e + e 2 Consider a vector u = a b = ae + be 2 How does f work on a general vector u?

28 Consider a linear map from R 2 to itself With the canonical basis of R 2 e = f : R 2 R 2 u! f (u), e 2 = We define f by the opera:on on the basis vectors: f (e ) = e, f (e 2 ) = e + e 2 Consider a vector ( ) = af (e )+ bf (e 2 ) f (u) = f ae + be 2 u = f is a linear map = ae + b(e + e 2 ) = (a + b)e + be 2 = a b = ae + be 2 a + b b = a b Representa:on of f(e i ) in the canonical basis: Rep( f (e )) =, Rep f (e 2 ) ( ) = A = We can define the linear map by this matrix

29 In general: f : R 2 R 2 We define the linear map on the basis vectors. The matrix representa:on of the linear map is given by the representa&on of the image of the basis vectors a a b f (e ) = ae + ce 2 = = c c d A = Rep b a b ( f (e )) Rep( f (e 2 )) f (e 2 ) = be + de 2 = = c d c d ( ) = a b d This defines the general form of the linear map as a matrix: u = u u 2 f (u) = f (u e + u 2 e 2 ) = u f (e )+ u 2 f (e 2 ) = u (ae + ce 2 )+ u 2 (be + de 2 ) = (au + bu 2 )e + (cu + du 2 )e 2 = a c b d u u 2

30 Consider a linear map between two vector spaces f : M N u! f (u) dim(m)=m, dim(n)=n Consider a basis in each vector space: B M = { u,...,u m }, B N = { v,..., v n } Every element of the basis B M has an image f(u i ) with a representa:on in B N n f (u i ) = a ji v j or Rep BN f (u i ) j= ( ) = (a i,..., a ni ) For every vector in M, with a representa:on in the basis B M u = α i u i f (u) = f m i= m m n α i u i = α i f (u i ) = α i a ji v j = i= i= j= m i= n α i a ji v j = j= n j= m i= m i= (*) a ji α i v j f is linear (*) associa:vity

m n m u = α i u i f (u) = a ji α i v j i= j= i= = ( a α + a 2 α 2 +... + a m α m )v +... + ( a n α + a n2 α 2 +.

31 m n m u = α i u i f (u) = a ji α i v j i= j= i= = ( a α + a 2 α a m α m )v ( a n α + a n2 α a nm α m )v n We obtain the matrix representa:on of a linear map using the representa:on of the linear map on the two basis: Basis from the origin space ( Rep BN (u )! Rep BN (u m ) ) Basis from the target space m α a! a m α a i α i i= f u =! f (u) = "! =! α m a n! a nm α m m a ni α i i= m nxm m n N

32 Example: f : R 2 R 3 We define f by: u! f (u) R 3 Consider the two bases: B R 2 = { u,u 2 }, B R 3 = { v, v 2, v 3 } f (u ) = v + 2v 2, f (u 2 ) = v v 3 f (u) = f ( au + bu 2 ) = af (u )+ bf (u 2 ) = av + 2av 2 + bv bv 3 = (a + b)v + 2av 2 bv 3 f (u) = 2 a b Rep( f (u )) = 2, Rep f (u 2 ) ( ) = A = 2

33 Example: f : R 2 R 3 f (e ) = (,, ) f (e 2 ) = (,, ) Consider the two bases for R 2 :,, B' = R 2 B R 2 = { e,e 2 } = { } B R 3 = e,e 2,e 3, 2, Matrix representa:on in the canonical basis: Rep( f (e )) =, Rep f (e 2 ) ( ) = A = The matrix representa:on depends on the basis: f (u) = f (ae' + be' 2 ) = af (e' )+ bf (e' 2 ) = af (e + e 2 )+ bf (e + 2e 2 ) = (a + b) f (e)+ (a + 2b) f (e 2 ) f (u) = Rep( f (e' )) Rep( f (e' 2 )) 2 a b

34 Given a linear map between two vector spaces f : M N u M! f (u) N dim(m ) = m, dim(n) = n We know there is a matrix representa:on for this linear map f (u) = Au, A M nxm (R) ( ) ( ) = dim( Ker( f )) + dim( Im( f )) In general rank(a) = dim Im( f ) dim M Recall that onto dim( Im( f )) = dim(n) = n If A squared and det(a) = rank(a) = dim(im( f )) < dim(n) f is not onto and dim(ker( f )) >

35 Consider a linear map from R 2 to itself with the canonical basis of R 2 f : R 2 R 2 e =,e 2 = We define f by: f (e ) = e 2, f (e 2 ) = Rep( f (e )) = Consider a vector, Rep( f (e 2 )) = a u = = ae + be 2 b A = f (u) = a b = a det(a) = rank(a) < 2 Ker( f ) = { v R 2 / f (v) = } = a b = = b, b R dim(ker( f )) = Im( f ) = { f (v) / v R 2 } = a, a R dim(im( f )) =

36 f : R 2 R 3 f (e ) = (,, ) f (e 2 ) = (,, ) Consider the two canonical bases B R 2 = e, e 2 { } B R 3 = e, e 2, e 3 { } Example: Matrix representa:on: Rep f (e ) ( ) =, Rep f (e 2 ) ( ) = A = rank(a) = 2 dim(im( f )) = 2 Ker( f ) = a b = = dim(ker( f )) = 2 = dim(r 2 ) = dim(im( f ))+ dim(ker( f ))

37 Example: Consider the two canonical bases f : R 3 R 2 f (e ) = (, ) f (e 2 ) = (,) f (e 3 ) = (,) B R 2 = { e, e 2 } B R 3 = { e, e 2, e 3 } Matrix representa:on: Rep( f (e )) =, Rep f (e 2 ) ( ) =, Rep f (e 3 ) ( ) = A = rank(a) = 2 dim(im( f )) = 2 < dim(r 3 ) Ker( f ) = a b c = = a a a, a R dim(ker( f )) = 3 = dim(r 3 ) = dim(im( f ))+ dim(ker( f )) = 2 +

38 Exercise Consider the following linear map between R 3 and the polynomials of degree f : R 2 P (R) where a u = b! f (u) = (2a + b) cx c Find a matrix representa:on for f P (R) = { αx + β, α, β R} Polynomials of degree

39 Exercise Consider the following linear map f : R 3 R 2 f (x, y, z)! (x + y, y + z) ) Find the associated matrix 2) Find the Kernel of f 3) Is f an isofomorphism?

40 Change of Basis Consider the vector space R 2 Consider two possible bases: E = e =, e 2 =, B = w =, w 2 = Consider a generic vector in R 2. We can represent this vector in the two bases: u = ae + be 2 Rep E (u) = u = cw + dw 2 Rep B (u) = But it is the same vector, so ae + be 2 = cw + dw 2 a b c d

41 Change of Basis Consider the vector space R 2 Consider two possible bases: E = e =, e 2 =, B = w =, w 2 = Consider a generic vector in R 2. We can represent this vector in the two bases: u = ae + be 2 Rep E (u) = u = cw + dw 2 Rep B (u) = But it is the same vector, so ae + be 2 = cw + dw 2 And we can write the vectors from B in terms of the vectors from E: w = e + e 2, w 2 = e e 2 We can rewrite this as follows: u = ae + be 2 = cw + dw 2 = c(e + e 2 )+ d(e e 2 ) = (c + d)e + (c d)e 2 a = c + d b = c d a = c b d a b c d

Change of Basis Let s write the vectors from B in terms of the vectors from E: w = e + e 2, w 2 = e e 2 We can the rewrite u as follows: Rep E (w ) =, Rep E (w 2 ) = u = ae + be 2 = cw + dw 2 = c(e +

42 Change of Basis Let s write the vectors from B in terms of the vectors from E: w = e + e 2, w 2 = e e 2 We can the rewrite u as follows: Rep E (w ) =, Rep E (w 2 ) = u = ae + be 2 = cw + dw 2 = c(e + e 2 )+ d(e e 2 ) = (c + d)e + (c d)e 2 a = c + d b = c d a = c b d Rep E (w ) Rep E (w 2 ) You can also write like this: Rep E (u) = a b = crep E (w )+ drep B (w 2 ) = c + d = c + d c d = c d We obtain a matrix opera:on to change from the representa:on in B to the representa:on in E

43 Change of Basis We can do the same for the inverse change: ae + be 2 = cw + dw 2 Rep B (e ) Rep B (e 2 ) Rep B (u) = c d Rep B (e ) = / 2 / 2 Rep B (e 2 ) = = arep B (e )+ brep B (e 2 ) = a / 2 / 2 / 2 / 2 + b / 2 / 2 = 2 = 2 = / 2 / 2 / 2 / 2 Where the representa:on of the canonical basis vectors e, e 2 in the new basis B is: because We have thus obtained a matrix opera:on to change basis between E and B: a b

44 Change of Basis We obtain an opera:on to change basis: ae + be 2 = cw + dw 2 From E à B From B à E c d = / 2 / 2 / 2 / 2 a b a b = c d Consider the vector v in the canonical basis: Rep E (v) = Rep B (v) = v = 2e + 2e 2 Rep B (v) = / 2 / 2 / 2 / 2 Rep E (v) = 2 2 = = 2 v = 2w v = e + 2e 2

45 Change of Basis The two matrices provide the transforma:on in opposite direc:ons, so as you would expect, one is the inverse of the other: / 2 / 2 / 2 / 2 = The matrix of change of basis can be understood as the matrix associated to the iden&ty linear map between two different bases: id : (R 2, E) (R 2, B) Rep E (u) = And the inverse map: a b! Rep B (u) = / 2 / 2 / 2 / 2 a b id : (R 2, B) (R 2, E) Rep B (u) = c d! Rep E (u) = c d

46 Change of Basis In general: id : (V, B) (V, D) Rep B (u) = u! u n n B = { β,..., β n }, D = { δ,...,δ n } u ( )! nxn " Rep D (u) = Rep D (β )... Rep D (β n ) Column vectors = representa:on of the old basis vectors in the new basis u n n The change of basis is represented as the iden:ty linear map of V onto itself id : (V, B) (V, D) Rep BD (id) = ( Rep D (β )... Rep D (β n ) ) nxn The matrix of the change of basis from B to D is the matrix formed of the representa:on in D of the basis vectors from B

47 Exercise Consider R 2 with the two following bases: E = e =, e 2 =, B = β = 2, β 2 = 2 3 Calculate the matrix of change of basis from E to B, and from B to E Consider the vector Calculate Rep B (v) v with representa:on in the canonical basis: Rep E (v) = 3 5

48 Composition and inverse Consider two linear maps: f :V W g :W U v V, A f v = w W w W, A g w = u U We can rewrite it as a composi:on: V f W g! f :V U g U This translates to a composi:on of matrices: v V, A g A f v = u U Composi:on is associa:ve but not commuta:ve: V f W h! g! f g U ( ) :V T ( h! g)! f :V T h T v V, A h ( A g A f )v = ( A h A g ) A f v T

49 Composition and inverse Defini&on: a linear map f :V W Is inver:ble if there is another linear map g :W V such that g! f = id V :V V f! g = id W :W W Iden:ty in V Iden:ty in W g is the inverse

50 Composition and inverse We have seen before that we can find a matrix representa:on of a linear map. This matrix representa:on depends on the bases used: Rep BB' (id) (V, B) id (V, B') A f (W, D) id f ' (W, D') A' Same liner map, but using bases B and D A' = Rep DD' (id) A Rep B'B (id) = Rep DD' (id) A ( Rep BB' (id)) f is a Linear map between V and W Rep DD' (id) We can change from one matrix representa:on to the other using the matrices for the change of basis:

51 Exercise Consider a linear map from R 2 to itself f : (R 2, E) (R 2, E) E = e = Using the canonical basis, the matrix representa:on is / 2 / 2 A f = / 2 / 2,e 2 = Transform this representa:on to another one with respect to other 2 bases: (R 2 f, E) (R 2, E) B = β = id (R 2, B), β 2 = 2 f ' (R 2, D) D = δ = id,δ 2 = 2 3 i.e. calculate the matrix for f : A' = Rep ED (id) A Rep BE (id) = Rep ED (id) A ( Rep EB (id))

Unit 5. Matrix diagonaliza1on

Unit 5. Matrix diagonaliza1on Linear Algebra and Op1miza1on Msc Bioinforma1cs for Health Sciences Eduardo Eyras Pompeu Fabra University 218-219 hlp://comprna.upf.edu/courses/master_mat/ We have seen before