Midterm for Introduction to Numerical Analysis I, AMSC/CMSC 466, on 10/29/2015 The test lasts 1 hour and 15 minutes. No documents are allowed. The use of a calculator, cell phone or other equivalent electronic device is not allowed. There are four exercises and 7 pages. Exercise 1. Give the LU decomposition of 1 4 4 A = 2 7 8. 1 5 6 Solution. Applying Gaussian elimination, one obtains 1 0 0 1 4 4 L = 2 1 0, U = 0 1 0. 1 1 1 0 0 2 1
Exercise 2. Let A be an upper triangular matrix of size N. How many multiplications/divisions are needed to solve Ax = y? Solution. Starting from the last row, one has x N = y N /A NN which requires 1 division. At row number i, one has x i = (y i N j=i+1 A ij x j )/A ii, that is 1 division and N i multiplications. This gives for all rows N divisions and N i=1 (N i) = N 1 k=0 k = N (N 1)/2 multiplications for a total of N 2 /2 + N/2. 2
Exercise 3. We wish to use Newton s method to find a root to 1. Prove that f has exactly one root. f(x) = x sin(x). Solution. First of all obviously f(0) = 0 so there is at least one root. Then f (x) = 1 cos x so f (x) 0 for all x and cannot vanish on an interval. Therefore f is strictly increasing and there is exactly 1 root. 2. Recall what Newton s method consists in. Solution. Newton s method consists in calculating a recursive sequence of approximations given by the following formula for a given starting point x 0. x n+1 = x n f(x n) f (x n ), 3
3. We define N f = x f(x). Calculate N f (x) f (x) and prove that it is continuous on R with N f (0) = 2. 3 Solution. N f is obviously well defined around 0 except at x = 0 since f (0) = 0. On the other hand, recalling that sin x = x x 3 /6 + O(x 5 ) and cos x = 1 x 2 /2 + O(x 4 ), one has that as x 0 N f (x) N f (0) x 0 = 1 x sin x x (1 cos x) = 1 x x + x3 /6 + O(x 5 ) x 3 /2 + O(x 5 ) = 1 x3 /6 + O(x 5 ) x 3 /2 + O(x 5 ) 1 2 6 = 2 3, proving that N f is differentiable at 0 with N f (0) = 2/3. 4. Deduce that there exists ε > 0 s.t. if x 0 < ε then the sequence defined by x k+1 = N f (x k ) converges to 0. Solution. We have just proved that N f is C 1 with N f (0) = 0 and N f (0) < 1. A theorem from class then implies that 0 is a stable point and that the sequence x k converges to 0 provided that x 0 is close enough. 4
5. What is the order of convergence? Solution. As N f (0) < 1 and N f (0) 0, the same result from class implies that the method is linear, order of convergence 1. This can also be recovered by calculating x k+1 x k = N f(x k ) x k = N f(x k ) 0 x k 0 N f(0) = 2/3. 5
Exercise 4 Consider A M N (R). 1. Assume that u is an eigenvector of A T A, i.e. A T A u = λu. Show that A u is an eigenvector of A A T if A u 0. Solution. One has that A T A u = λ u and so multiplying by A on the left of the equality A A T A u = A A T (A u) = λ A u. This proves that A u is an eigenvector provided that it is different from 0 of course. 2. Deduce that A T A and A A T have the same eigenvalues if A is invertible. What could happen if A is not invertible? Solution. As a symmetric matrix, A T A is diagonalizable in an orthonormal basis e 1... e N with corresponding eigenvalues λ 1... λ N. If A is invertible then A e 1,..., A e N is also a basis giving N eigenvectors for A A T with the same eigenvalues. Hence A A T and A T A have the same eigenvalues. If A is not invertible then one can still diagonalize A T A on e 1... e N. A e i will be an eigenvector for A A T unless A e i = 0 which means A T A e i = 0 and so λ i = 0. Let us thus separate the eigenvectors and eigenvalues of A T A: Assume that e 1... e k correspond to the null space of A T A while e k+1... e N all have eigenvalues λ i 0. Now A e k+1... A e N are linearly independent; otherwise for some linear combination u, one would have A u = 0 which implies A T A u = 0 whereas u cannot belong to the null space of A T A (it is a linear combination of independent vectors orthogonal to the null space). Consequently A A T also has at least N k eigenvectors with non 0 eigenvalues. On the other hand, one can make the same argument by diagonalizing A A T and using A T instead of A. Therefore A A T and A T A have exactly the same non 0 eigenvalues (with the same multiplicity). Since they both must have N eigenvalues in total, they actually still have the same spectrum. 6
3. Assume that A is diagonalizable and invertible and denote by λ M the largest eigenvalues of A in absolute value. Prove that A 2 λ M, and deduce that the largest eigenvalue of A T A is larger than λ 2 M. Give an example where A 2 > λ 2 M. Solution. Denote by u the eigenvector associated with λ M and observe that A 2 = max x 0 A x 2 x 2 A u 2 u 2 λ M. Now denote by Λ the largest eigenvalue of A T A. We have seen in class that A 2 = Λ which implies that Λ λ 2 M. Finally it is sometimes possible to have a strict inequality. Take for example [ ] 1 1 A =, 0 1 with hence λ M = 1. But then [ ] 1 0 A T =, A T A = 1 1 [ ] 1 1, 1 2 with characteristic polynomial (1 λ)(2 λ) 1 = λ 2 3λ + 1 with Λ = (3 + 5)/2 > 1. 7