Mth 520 Finl Exm Topic Outline Sections 1 3 (Xio/Dums/Liw) Spring 2008 The finl exm will be held on Tuesdy, My 13, 2-5pm in 117 McMilln Wht will be covered The finl exm will cover the mteril from ll of the lectures nd redings, i.e. chpters 1-7 of the textbook, excluding the pseudo-inverse (in section 7.4). Approximtely hlf of the exm will be devoted to mteril covered fter the second midterm. Outline of topics covered fter Exm 2 (1) Digonlizbility ( 6.2) () We sy A is digonlizble if it hs n linerly independent eigenvectors. Equivlently, there is bsis of R n consisting of eigenvectors of A. (b) Some mtrices re digonlizble (like ( 2 1 1 2 )), while others re not (like ( 2 0 1 2 )). (c) A rndom mtrix is likely to be digonlizble, nd ny mtrix cn be mde digonlizble by chnging its entries by n rbitrrily smll mount. (d) If A hs n distinct eigenvlues, then A is digonlizble; this follows from two fcts: (i) Every eigenvlue hs t lest one ssocited eigenvector (ii) Eigenvectors with different eigenvlues re linerly independent (e) If λ is n eigenvlue of A, its lgebric multiplicity is the number of times it ppers s root of det(a λi). For exmple, if det(a λi) = λ 2 (λ + 1)(λ 2) 3 then the lgebric multiplicities re: λ AM 0 2-1 1 2 3 (f) If λ is n eigenvlue of A, its geometric multiplicity is the mximl number of linerly independent eigenvectors with eigenvlue λ, i.e. GM = dim N(A λi). For exmple, here re two 2 2 mtrices whose only eigenvlue is 2 (hence AM = 2), but where the geometric multiplicities differ. A GM of λ = 2 ( 2 0 0 2 ) 2 ( 2 0 1 2 ) 1 (g) A is digonlizble exctly when AM = GM for ech eigenvlue. Since GM AM, non-digonlizble mtrices re exctly those with too few eigenvectors.
2 (2) Digonliztion ( 6.2) () Suppose A is digonlizble mtrix with eigenvlues λ 1,..., λ n nd ssocited eigenvectors x 1,..., x n, i.e. Ax i = λ i x i. (b) Let λ 1 λ 2 Λ =.... λ n This is the digonl eigenvlue mtrix. (c) Let S = x 1 x 2 x n This is the eigenvector mtrix, which is invertible becuse its columns form bsis of R n. (d) Since Ax i = λ i x i, we hve A = SΛS 1. (e) Conversely, if A = SΛS 1, then the columns of S re eigenvectors nd the digonl entries of Λ re the eigenvlues. (f) Since A k = SΛ k S 1, the mtrix A k hs the sme eigenvectors s A, with eigenvlues λ k 1,..., λk n. (g) If 0 is not n eigenvlue, then A is invertible nd A 1 = SΛ 1 S 1, so A 1 hs the sme eigenvectors s A, with eigenvlues 1/λ i,..., 1/λ n. (h) A k 0 s k if ll of the eigenvlues of A stisfy λ i < 1. (This is true even if A is not digonblizble, but for digonlizble mtrices it follows from A = SΛS 1.) (3) Appliction: Difference Equtions ( 6.2) () Define sequence of vectors strting with u 0 nd pplying the rule u k+1 = Au k. (b) If x i is n eigenvector of A with eigenvlue λ i, then u k = λ k i x i is solution of u k+1 = Au k. (c) If λ i < 1, then the corresponding solution decys s k, while if λ i > 1 the solution blows up. (d) If λ i = 1, then ny multiple of x i is stedy-stte solution, becuse Ax i = x i. (e) If A is digonlizble, then its eigenvectors form bsis of R n, so ny strting vector u 0 is liner combintion of them: u 0 = c 1 x 1 +... c n x n
3 The corresponding solution to u k+1 = Au k is therefore u k = c 1 λ k 1x 1 +... c n λ k nx n (f) The mtrix form of this solution is u k = A k u 0 = SΛ k S 1 u k = SΛ k c. (g) This method llows us to find n explicit formul for the Fiboncci numbers (0, 1, 1, 2, 3, 5, 8, 13, 21, 33,...), which obey the second-order reltion F k+2 = F k+1 + F k. This cn be mde into first-order difference eqution by considering the vector Fk+1 u k = F k which stisfies u k+1 = 1 1 u 1 0 k. (h) The k th Fiboncci number is pproximtely ( F k 1 1 + ) k 5 5 2 becuse 1+ 5 2 is the only eigenvlue of ( 1 1 1 1 0 ) lrger thn 1. (The fctor 5 comes from the initil conditions F 0 = 0, F 1 = 1.) (4) Appliction: Differentil Equtions ( 6.3) () Given n n n mtrix A, consider the system of differentil equtions where du dt = Au u 1 (t) u(t) =.. u n (t) This system is liner, first-order, nd hs constnt coefficients. (b) If x i is n eigenvector of A with eigenvlue λ i, then u(t) = e λ it x i is solution of du dt = Au. It is pure exponentil solution, where u(t) moves long the line of multiples of x i. (c) If Re(λ i ) < 0, then the corresponding solution decys s t, while if Re(λ i ) > 0 the solution blows up. (d) If λ i = 0, then ny multiple of x i is stedy-stte solution, becuse Ax i = 0.
4 (e) If A is digonlizble, then its eigenvectors form bsis of R n, so ny initil condition u(0) is liner combintion of them: u(0) = c 1 x 1 +... c n x n The corresponding solution to du dt = Au is therefore u(t) = c 1 e λ 1t x 1 +... c n e λnt x n (f) The mtrix form of this solution is u(t) = e At u(0) = Se Λt S 1 u(0) = Se Λt c. (g) The mtrix exponentil e A is defined by e A = I + A + 1 1 2 A2 + = k! Ak k=1 nd this sum converges for ll n n mtrices A. (h) The exponentil of digonl mtrix is e 0 B @ λ 1 λ2... λ n 1 C A = e λ 1 e λ 2.... e λn (i) Higher-order liner equtions with constnt coefficients cn be reduced to systems of first-order equtions; for exmple, to solve the eqution y + by + ky = 0 we consider the vector function ( y u(t) = ) (t) y(t) which stisfies du dt = b k u. 1 0 (j) For 2 2 mtrix A, the solutions of du dt = Au will be stble (decy s t ) if the trce of A is negtive nd the determinnt of A is positive: tr(a) < 0 nd det(a) > 0 (5) Symmetric Mtrices nd the Spectrl Theorem ( 6.4) () Every symmetric mtrix A is digonlizble. (b) The eigenvlues of symmetric mtrix re rel, nd its eigenvectors re orthogonl. (c) If the eigenvectors re chosen to hve unit length, they form n orthonorml bsis of R n, so the eigenvector mtrix Q is orthogonl. Thus digonliztion becomes A = QΛQ T for symmetric mtrix A, which is the spectrl theorem.
(d) The spectrl theorem llows us to write A s combintion of projections, A = λ 1 P 1 + + λ n P n where P i is the projection onto the spn of the eigenvector q i. (e) The pivots nd the eigenvlues of symmetric mtrix re different, but re both rel. Furthermore, the number of positive (resp. negtive) pivots is equl to the number of positive (resp. negtive) eigenvlues. The number of zero eigenvlues is the dimension of the null spce (which one might be tempted to cll the number of zero pivots ). (6) Positive Definite Mtrices ( 6.5) () A mtrix is positive definite if it is symmetric nd its eigenvlues re positive, i.e. λ i > 0. (b) A symmetric mtrix is positive definite if nd only if its pivots re positive. This is the pivot test, which works becuse the pivots nd eigenvlues of symmetric mtrix hve the sme signs. (c) A symmetric mtrix A is positive definite if nd only if its upper-left determinnts d 1,..., d n re positive, where d 1 = det( 11 ) = 11 11 d 2 = det 12 21 22. 11 1,(n 1) d n 1 = det.. (n 1),1 (n 1),(n 1) d n = det(a) This is the determinnt test, which works becuse the pivots re rtios of these determinnts. (d) A symmetric mtrix A is positive definite if nd only if x T Ax > 0 for ll nonzero vectors x. This is the qudrtic form test. (e) The quntity x T Ax is qudrtic form, mening tht it is qudrtic expression in the components of x; specificlly, x T Ax = i,j x 1 ij x i x j where x =. (f) The qudrtic form test works becuse x T Ax cn be written s sum of squres where the coefficients re the pivots; this is clerly positive if ll of the pivots re positive, nd cn be zero or negtive otherwise. x n. 5
6 (g) For 2 2 symmetric mtrix A, we hve the LDL T decomposition b 1 0 0 1 b = b c 0 1 b 1 0 c b 2 so the pivots re nd c b2. Similrly, the qudrtic form x T Ax is sum of squres: ( x T Ax = x 2 1 + 2bx 1 x 2 + cx 2 2 = x 1 + b 2 2) x + c b 2 (x 2 ) 2 (h) If A is positive definite, then the grph of x T Ax is prboloid. Its minimum is t x = 0. (i) If A hs both positive nd negtive eigenvlues, then the grph of x T Ax is sddle-shped. (j) If A is positive definite, the eqution x T Ax = 1 defines n ellipsoid in R n. The principl xes of the ellipsoid re the lines contining the eigenvectors q 1,..., q n of A, nd the hlf-xis lengths re 1/ λ i. This comes from A = QΛQ T. (k) Thus one cn drw the ellipse defined by x 2 + 2bxy + cy 2 by finding the eigenvlues nd eigenvectors of A = b b c. (l) Positive definite mtrices generlize the second-derivtive test from clculus: A criticl point of rel-vlued function f(x 1,..., x n ) is minimum if the Hessin mtrix is positive definite. lwys symmetric. 2 f H ij = x i x j Becuse prtil derivtives commute, the Hessin is (7) Similr Mtrices nd Jordn s Theorem ( 6.6) () Two n n mtrices A nd B re similr if B = M 1 AM for some invertible mtrix M. (b) Similrity is n equivlence reltion, mening it is (i) Symmetric: A similr to B iff B similr to A (ii) Trnsitive: A similr to B nd B similr to C implies A similr to C (c) A is digonlizble if nd only if it is similr to digonl mtrix Λ. (d) Similr mtrices hve the sme: (i) eigenvlues, (ii) lgebric nd geometric multiplicities, (iii) trce, nd (iv) determinnt. (e) The eigenvectors of A nd of B = M 1 AM re different, but relted; if x is n eigenvector of A with eigenvlue λ, then M 1 x is n eigenvlue of B with eigenvlue λ. (f) If A hs distinct eigenvlues λ 1,..., λ n, then its similrity clss (i.e. the set of ll mtrices similr to A) consists of ll mtrices with eigenvlues λ 1,..., λ n.
(g) If A hs repeted eigenvlue, then nother mtrix with the sme eigenvlues my or my not be similr to A. For exmple, 2 1 2 0 A = nd B = 0 2 0 2 hve only one eigenvlue (λ = 2) but re not similr. In fct, B is not similr to ny mtrix except itself. (h) Even if A nd B hve the sme eigenvlues nd the sme number of eigenvectors, they my not be similr. For exmple 0 1 0 0 0 1 0 0 A = 0 0 0 0 0 0 0 1 nd B = 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 hve zero s their only eigenvlue, ech hs two independent eigenvectors, but A 2 = 0 while B 2 0 so they re not similr. (i) To clssify the different similrity clsses of mtrices, we find good representtive of ech clss, generlizing the digonl mtrix Λ similr to ny digonlizble mtrix. (j) The k k Jordn block J k (λ) hs λ s its only eigenvlue, nd hs one eigenvector: λ 1...... J k (λ) =... 1 λ 7 (k) Jordn s Theorem sys tht every mtrix A is similr to block-digonl mtrix J of Jordn blocks. Furthermore J is unique up to re-ordering the blocks, nd is clled the Jordn form of A. The numbers λ i in on the digonls of the blocks re the eigenvlues of A, while the number of blocks is the number of eigenvectors. (l) A is digonlizble if nd only if its Jordn form hs n blocks, ech of which must then hve size 1 1. (8) Singulr Vlue Decomposition ( 6.7) () If S is symmetric mtrix, then S = QΛQ T, so S preserves set of orthogonl coordinte xes. Most liner trnsformtions do not hve this property. The SVD ttempts to do something similr for more generl mtrices. (b) For generl squre mtrix A, one cn find n orthonorml bsis v 1,..., v n such tht Av 1,... Av n is n orthogonl bsis. (c) For such v 1,..., v n, we hve nother orthonorml bsis consisting of u 1 = 1 σ 1 Av 1, u 2 = 1 σ 2 Av 2,..., u n = 1 σ n Av n where σ i = Av i. These numbers σ i re the singulr vlues of A. (d) The singulr vlues of A re not the sme s the eigenvlues of A.
8 (e) If U is the mtrix with columns u i, nd V is the mtrix with columns v i, then both re orthogonl, nd we hve A = UΣV T where Σ is the digonl mtrix with digonl entries σ 1,..., σ n. This is the singulr vlue decomposition. (f) Finding the SVD: The vectors v i re the eigenvectors of A T A, nd σi 2 re the eigenvlues. Similrly, the vectors u i re eigenvectors of AA T, with the sme eigenvlues σi 2. (g) Geometriclly, the singulr vlues re the hlf-xis lengths of the ellipsoid {Ax x = 1}. (9) Liner Trnsformtions ( 7.1-7.3) () A mp T : V W (where V nd W re vector spces) is liner trnsformtion if both of these conditions hold: (i) T (cv) = ct (v) for ll c R nd v V (ii) T (v 1 + v 2 ) = T (v 1 ) + T (v 2 ) for ll v 1, v 2 V (b) A liner trnsformtion is lso clled liner mp, or liner mpping, or we my simply sy tht T is liner. (c) A liner mp T stisfies T (0) = 0 (d) Exmples of liner mps include: (i) T : R n R m defined by T (x) = Ax where A is n m n mtrix (ii) The identity mp Id : V V, defined by Id(v) = v for ll v (iii) The zero mp Z : V W, defined by Z(v) = 0 for ll v (iv) A rottion R : R 2 R 2 of the plne bout the origin (v) Projection P : R 2 R 2 of the plne onto line through the origin (e) The following mps re not liner: (i) The determinnt mp, det : M n n R (ii) The length mp L : R n R, where L(v) = v (iii) A shift mp T : V V, where T (v) = v + v 0 nd v 0 0 (f) If v 1,..., v n is bsis of V, then ny vector v V cn be expressed s v = c 1 v 1 +... c n v n for unique rel numbers c 1,..., c n. These numbers re the coordintes of v with respect to the bsis v 1,..., v n. (g) If v 1,..., v n is bsis of V, then ny liner mp T : V W is uniquely determined by the vectors T (v 1 ),..., T (v n ) W (h) Choosing n input bsis v 1,..., v n of V nd n output bsis w 1,..., w m of W llows us to ssocite mtrix A to liner trnsformtion T ; the
9 entries of A re the coordintes of T (v i ): T (v 1 ) = 11 w 1 + 21 w 2 + + m1 w m T (v 2 ) = 12 w 1 + 22 w 2 + + m2 w m. T (v n ) = 1n w 1 + 2n w 2 + + mn w m Thus A is n m n mtrix whose j th column gives the coordintes of T (v j ) with respect to w 1,..., w m. 11 1n A =.. m1 mn (i) If T : V V is liner trnsformtion from vector spce to itself, nd v 1,..., v n is bsis of eigenvectors, then the mtrix ssocited to T (using v 1,..., v n s both the input nd output bsis vectors) is digonl. (j) Stndrd bsis of R n is e 1,..., e n where 1 0 e 1 = 0 e 2 = 1 etc... (k) Chnge of bsis for R n : If w 1,..., w n form bsis of R n, put these into the columns of mtrix W. Then the mtrix W 1 trnsforms vector into its coordintes with respect to w 1,..., w n. Tht is, if v = c 1 w 1 + c 2 w 2 + c n w n then 1 c. = W 1 v c n (l) We cn regrd W s the mtrix of the identity trnsformtion with input bsis e 1,..., e n nd output bsis w 1,..., w n. (m) If w 1,..., w n nd z 1,..., z n re bses of R n, then Z 1 W chnges coordintes reltive to w i into coordintes reltive to z i. Red this s from w i to stndrd, then from stndrd to z i. Thus M = Z 1 W is chnge of bsis mtrix. (n) This chnge of bsis mtrix M is the mtrix of the identity liner trnsformtion, with input bsis w 1,..., w n nd output bsis z 1,..., z n. (o) If A nd B re mtrices representing the sme liner trnsformtion T : V W, then A = W 1 BW 2 where W i re chnge of bsis mtrices. (p) If A nd B re mtrices representing T : V V, nd in ech cse, the output nd input bses re the sme, then A nd B re similr.