Homework Haimanot Kassa, Jeremy Morris & Isaac Ben Jeppsen October 7, 004 Exercise 1 : We can say that kxk = kx y + yk And likewise So we get kxk kx yk + kyk kxk kyk kx yk kyk = ky x + xk kyk ky xk + kxk kyk kxk ky xk kxk kyk kx yk Exercise : (An Induced Matrix Norm.) Let A be a real n x n matrix. Show that: kak 1 = max ;;:::;n ja ij j: Hint: Use the denition of the induced matrix norm: kak 1 = max kaxk 1 : kxk 1 =1 Answer: let kxk 1 = jx j j 1
then kaxk 1 = j a ij x j j ja ij jjx j j: Changing the order of summation, we can separate the summands, kaxk 1 ja ij jjx j j: let then c = max 1jn ja ij j (1) kaxk 1 ckxk 1 and thus kak 1 c to show this as an equality, we demonstrate an x for which kaxk 1 kxk 1 = c let k be the column index for which the maximum in (1) is attained. Let x = e k, the k th unit vector. Then kxk 1 = 1 and kak 1 = j a ij x j j = ja ik j = c This proves that for the vector norm k k, the operator norm is kak 1 = max 1jn ja ij j Exercise : An induced norm is dened as kak = max kaxk kx
if we let A = I n then but kak F = kik F = vu u t n X a ij = p 1 + 1 + : : : + 1 = p n kak = kik = max kixk = max kxk = 1 kx kx Since max kaxk 6= kaxk F the Frobenius norm is not an induced norm. kx Exercise 4 : Does the spectral radius itself dene a norm? Why or why not? Answer. As we have seen it in class, for an arbitrary square matrix A, r (A) kak () Moreover, if " > 0 be given, then there is an operator matrix norm, for which kak " r (A) + " () This shows that r (A) is almost a matrix norm. But notice that it does not satisfy all the norm properties. For example kak = 0 $ A = 0 but this is not neccessarly true for spectral radius of a matrix. Take this example: A = 4 0 0 0 1 0 0 0 5 (4) The spectral radius of A is 0, but A is not the zero matrix. Exercise 5 : Derivation: Given Ax = b we want to obtain an approximation ex to x, ~x 6= x. This leads to kek = k~x xk kek ( relative error) kxk Which leads to the following set of equations: Ax = b! kbk 6 kakkxj (1) A 1 b = x! kxk 6 ka 1 kkbk () Ae = r! krk 6 kakkek () A 1 r = e! kek 6 ka 1 kkrk (4)
Dividing the smaller side of (4) by the larger side of (1), and the larger side of (4) by the smaller side of (1) gives Similarly with () and (): kek kxk 6 kakka 1 k krk kbk krk ka 1 kkbk 6 kakkek kxk Which leads to 1 krk kakka 1 k kbk 6 kek kxk 6 kakka 1 k krk kbk In general this inequality can be shown to be sharp trivialy by denition. That is given kak = max kaxk kx By denition, there is some x such that kbk = kaxk = kakkxk Similarly for equations (),(),(4). Therefore there is some x such that for the right hand side we have kek kakkxk = ka 1 k krk kbk And for the left hand side we have: krk ka 1 kkbk = kakkek kxk or a) For the more specic case of k k : Using SVD kaxk = kbk becomes: Let x = v 1 then kek or kxk = kakja 1 k krk kbk krk ka 1 kkakkbk = kek kxk (i) kaxk = kuv T xk kaxk = kuv T v 1 k = (UV T v 1 ) T (UV T v 1 ) = v T 1 V T U T UV T v 1 4
Since U and V are orthogonal this becomes T = kk By denition of the induced matrix norm this becomes r n kk = max kxk orkk = max kxk =1 kxk =1 Since by denition 1 = max( i ), And since kxk = 1. r n kxk = 1 max x i = kxk 1 =1 i x i Similarly for ka 1 rk = kek letting x = u i yields: r n ka 1 rk = ka T u i k = kv 1 U T k = n 1 max x i = 1 n Similar steps for equations () and () yield kxk =1 Ax = b! kbk = kaxjj = 1 kxk (1.a) A 1 b = x! kxk = ka 1 bk = 1 n kbk (.a) Ae = r! krk = kaek = 1 kek (.a) A 1 r = e! kek = ka 1 rk = 1 n krk (4.a) Which gives for the right hand side: and for the left hand side: kejj kxk = 1 krk n kbk krk 1 n 1 kbk = kek kxk Therefore the inequality is sharp when A is a diagonal matrix. b) For k k 1 By denition: kak 1 = max kaxk 1 = max kx i 5 ja ij x j j 6 max i ja ij j = ja kj j
Supposing that the max row sum is obtained in the k th row, pick x to be 1 based on the signs of this k th -row. x = 0 B @ + 1 + 1 : : : + 1 1 C A Given this x the inequality (i) becomes an equality P Ax = b! kbk = max kaxjj 1 = n ja kj j = kx P A 1 b = x! kxk = maxka 1 bk1 = n kb Ae = r! krk = max ke ja 1 kj j = P kaek= n ja kj j = P A 1 r = e! kek = max ka 1 rk 1 = n kr ja 1 kj j = (1.a) (.a) (.a) (4.a) And thus, for the right hand side: kejj kxk = krk kbk And for the left hand side: krk kbk = kek kxk Exercise 6 : General case A is the Hermitian (complex symmetric matrix) i) If A = A then so hax; xi is real hax; xi = hx; Axi = hx; Axi = hax; xi ii) If Ax = x, then hax; xi = hx; xi = hx; xi. Since hax; xi and hx; xi are real must be real. 6
It follows directly that A = A T is a special case of this. Therefore, the eigenvalues of A are real. Exercise 7 : Let be an eigenvalue of AA nxn. Show that there exists an such that i f1; ; : : : :; ng ja ii j 6 Xn jaij j Proof: Let x i be the component of largest magnitude of one of the eigenvectors of A. From the ith equation of the system (A I)x = 0, we have Which leads to: (a ii )x i = ja ii j 6 j6=i j6=i j6=i ja ij j jx jj jx i j 6 a ij x j Let S be a set that is the union of k 6 n Gershgorin circles s.t. the intersection of S with all other Gershgorin circles is empty. Show that S contains precisely k eignvalues of A (counting multiplicities) Let k = n, this implies that A is a diagonal matrix, A = D, This yields n circles centered about the diagonal entries of a ii with radius 0. Give an example showing that on the other hand there may be Gershgorin circles that contain no eigenvalues of A at all. Let: 1 1 A = 1 4 The eigenvalues for this matrix are = 1:908 : : : ; 0:596 : : : which are both outside the circle of radius 0.5 centered at 1. NOTE: A woman by the name of Olga Taussky Todd was famous for using this theorem and worked with the utter group to help build more ecient aircrafts during WWII. The utter speed is extremely important to an aircrafts making and must be carefully calculated for it to be able to get o the ground. 4 j6=i ja ij j 7
Olga found a more simplied way to nd these calculations by nding the eigenvalues and eigenvectors using the Gershgorin Theorem. \Still, matrix theory reached me only slowly," Taussky noted in a 1988 article in the American Mathematical Monthly. \Since my main subject was number theory, I did not look for matrix theory. It somehow looked for me." (IvarsP Peterson's MathTrek). Exercise 8 : Let A R ml and B R ln and show that if we use partitioned matrices A11 A A = 1 B11 B B = 1 A 1 A B 1 B Where A ij R m il j and B ij R l in j. Then the matrix product can be computed using: A11 B AB = 11 + A 1 B 1 A 11 B 1 + A 1 B A 1 B 11 + A B 1 A 1 B 1 + A B To show this, we will take the statement for the upper left block matrix AB and expand it: (AB) ij = l 1 X (a 11 ) ik (b 11 ) kj + l X (a 1 ) ik (b 1 ) kj (5) Which can be written in terms of the original matrices A and B Xl 1 a ik b kj + lx 1 +l k=l 1 +1 And we can combine these sums by writing In general we can write lx 1 +l (AB) ij = a ik b kj (6) a ik b kj (7) A^ik B k^j (8) where n is the number of row matrices of A and the number of column matrices of B. And if A R ml with sub-matrices A ij R m il j where 8
l 1 + l + : : : + l n = l and, ^i, ^j refer to the appropriate matrices. Exercise 9 : a. False. By denition, the determinant is the sum of all possible products where we pick an element from each row and column of the matrix. This means that if we take a matrix A R 44 4, that we should have 4 = 4 products to sum. However, by this method we only get 8. det (A) = det (A 11 ) det (A ) det (A 1 ) det (A 1 ) = a 11 a a a 44 a 11 a a 4 a 4 a 1 a 1 a a 44 + a 1 a 1 a 4 a 4 a 1 a 4 a 1 a 4 + a 1 a 4 a a 41 + a 14 a a 1 a 4 a 14 a a a 41 Notice that there are only two terms with the coecient a 11, if we compute the determinant using the minor expansion, we should have six such terms. b. False, take the matrix Where we have sub-matrices A = 6 4 B = 1 C = D = 1 0 0 0 1 0 0 1 0 4 0 4 0 1 0 0 4 4 0 1 0 Then we have rank(b) + rank(d) = but rank(a) =. This statement could be true if rank(b) = rank(c). Exercise 10 : To count the number of multiplications and divisions in the Cholesky decomposition, we use the equations derived in the book for nding the individual elements of the matrix L l ij = a ij 9 j 1 X l jj 0 5 7 5 5 l ik l jk
l ii = " a ii Xi 1 l ik # 1= The number of divisions can be seen from the rst equation, there are divisions for every element below the diagonal. This is n n To get the number of multiplications we notice that we need the double sum X X n 1 n 1 j=i n j Using a method similar to the one used in the notes, we approximate this expression with integrals to get the number of multiplications Z n 1 Z n 1 1 i (n j) dj di = 1 6 n 1 n + For the case when i = j, we use the second equation to get the sum (n i) Z n 1 (n i) di = n n + 1 When we approximate using integrals, the leading term is the only one that is important. Therefore, we see that the Cholesky decomposition is of order n 6 10