12. Perturbed Matrices

Transcription

1 MAT334 : Applied Linear Algebra Mike Newman, winter Perturbed Matrices motivation We want to solve a system Ax = b in a context where A and b are not known exactly. There might be experimental errors, so our solution x represents the correct answer to what is technically the wrong question. We will consider A and A, two matrices that are close to each other, and two vectors b and b that are close to each other. We will sometimes write A A, b = b b, etc. The notation does carry one risk of confusion: A is one matrix, and not a scalar multiplied by the matrix A. We typically think of A and b as being small. The question comes down to the following: If x is a solution to Ax = b, is it close to a solution for A x = b? We will start by considering invertibility of matrices. invertibility near the identity If a matrix is close to the identity matrix, we would like to think that it resembles it (even though it might not be similar!). A matrix A close to the identity matrix can be understood in terms of P = A I, or I + P, for some small matrix P. We need to better understand the meaning of small. Theorem 2.. Let P be an n n matrix and any matrix norm. If P < then I + P is invertible and furthermore + P (I + P ) P. Proof. Recall that a matrix is invertible if and only if its null space is trivial (see Theorem.23). So to show that I + P is invertible we will show that (I + P )x = 0 implies x = 0. If (I + P )x = 0 then P x = Ix = x. This gives x = x = P x P x. If x 0 then x = 0 and hence P, which is a contradiction. So the only possibility is that x = 0 and thus I + P is invertible. In order to derive the approximation for (I + P ), set B = (I + P ). Since I = (I + P )B we get ( ) = I = (I + P )B I + P B I + P B Since B = I P B we get Solving these for B we get the desired result. B = I P B I + P B + P B Problem 2.2. We said I P B I + P B. Why is it + and not? These notes are intended for students in mike s MAT334. For other uses please say hi to mnewman@uottawa.ca. 88

2 This result tells us that matrices close to the identity are invertible. A small change preserves the property of invertibility., Example 2.3. Let What does Theorem 2. tell us about A? We see that I + P with P = We calculate P = max {0.2 ;. ; 0.2} =.. Using this norm, P and so Theorem 2. doesn t apply with this norm. We can also calculate P = max {0.5 ; 0.5 ; 0.5} = 0.5. Since P <, Theorem 2. applies with this norm, and we conclude that I + P is invertible. Furthermore we know that 2/3 A 2. This guarantees that the sum of the absolute values in each row of the inverse is at most 2, and that in one of the rows this sum is at least 0.5. In particular the absolute value of each entry in the inverse is at most 2, and at least one entry has absolute value at least 2/3. Problem 2.4. Let a C with a <, and P = 0 a. a 0 Show that P = P = P 2 = a. Is I + P invertible? Give bounds on the norms of (I + P ) that follow from Theorem 2., and interpret these. Find the inverse (I + P ) and its norm, and compare with the bounds obtained from Theorem 2.. inverses Theorem 2. explains what happens to matrices close to the identity matrix. extend these ideas to matrices close to any arbitrary invertible matrix. We would like to Theorem 2.5. Let A and R n n matrices with A invertible, and any norm. Let α = A R or α = RA. A If α < then A + R is invertible and furthermore + α (A + R) A α. Note that we have two choices for α. In general A R RA. A smaller value of α gives a stronger theorem, so we would choose the smaller of the two options. Proof. Assume that α = A R <. The matrix A is invertible, so we can set P = A R. We see that A + R = A(I + P ) and P = α <, so by Theorem 2. the matrix I + P is invertible. The product of invertible matrices is invertible and so A + R is invertible in fact (A + R) = (I + P ) A. In order to bound the norm, we again appeal to Theorem 2.. A = (I + P )(A + R) (I + P ) (A + R) ( I + P ) (A + R) (A + R) = (I + P ) A (I + P ) A A P Solving these for (A + R) we get the desired result. 89

3 Problem 2.6. Show the case of α = RA of Theorem ɛ Example 2.7. Let, for ɛ > 0. This is an invertible matrix. For which matrices ɛ R can we guarantee that A + R is also invertible? We see that A = (2 + ɛ)/ɛ 2. Since A R A R, having R < ( A ) is sufficient to guarantee A R <. Thus if R < ɛ2 2 + ɛ then A + R is invertible. If ɛ is small, then to guarantee that A + R is invertible we would need to have R very small. Problem 2.8. Find a matrix R with A + R non-invertible. Show that your example is consistent with the condition given above. + ɛ Problem 2.9. Find the inverse of, and show that A ɛ = (2 + ɛ)/ɛ 2. Also, find A and A 2, and give a condition on R and R 2 that guarantees that A+R is invertible. We see an important detail in Example 2.7. If we want to know if A + R is invertible for some fixed matrix A and an error term R, a condition on R is more practical than a condition on A R. We often use this weaker form of Theorem 2.5. Theorem 2.0. Let A and R be n n matrices with A invertible, and any norm. Let α = R A. A If α < then A + R is invertible and furthermore + α (A + R) A α. Proof. We see that RA } A R R A < ( A ) A = Then Theorem 2.5 guarantees that A + R is invertible. The condition R < ( A ) can be rewritten in the following seemingly more complicated way. R < A If we think of A as an initial matrix and A + R as the perturbed matrix, then R is an error term, and R / is the relative perturbation, or relative error. This equation says that the relative error is bounded by a nice function of the matrix and its inverse. Motivated by this, we define the condition number of a matrix A as c(a) = A. Notice that the condition number of a matrix is relative to the choice of norm. 90

4 Corollary 2.. Let A an R be n n matrices with A invertible, and any norm. Let α = R c(a). If α < then A + R is invertible and furthermore A + α (A + R) A α. Proof. This is just Theorem 2.0 rewritten. Lemma 2.2. For any invertible matrix A, c(a). Proof. exercise! Problem 2.3. Let and R =. 0 0 Observe that A + R is not invertible, and calculate R /. Calculate A,, A and thus c(a). Check that c(a) R / and explain how this agrees with Theorem 2.0 and Theorem 2.5. Redo this exercise for the norm. linear systems We want to apply the idea of Theorem 2.5 to the solution of linear systems. Let Ax = b be a linear system we want to solve, and A x = b a close system. We set A = A + A, b = b + b and x = x + x. So A and b are small matrices and x is the corresponding change in the solution. For instance, we could have obtained this system from some experimental data, for which the values are not exactly known. We can solve the system, but we want to know to what extent the experimental errors cause errors in the solution. Briefly we want to know to what extent x is small, given that A and b are small. Theorem 2.4. Let Ax = b where A is an invertible matrix and b, x are nonzero. Let A x = b, and define A A, b = b b, x = x x. Let α = ( A)(A) or α = (A) ( A). If α < then: x x α c(a) ( b b + A ) Proof. Theorem 2.5 guarantees that A+ A is invertible, since α <. We can write x in terms of the other matrices. (A + A)(x + x) = (b + b) (A + A) x = (b + b) (A + A)x (A + A) x = b + b Ax Ax (A + A) x = b Ax x = (A + A) ( b Ax) 9

5 This allows us to get an upper bound on x. Theorem 2.5 is again useful. x = (A + A) ( b Ax) (A + A) ( b Ax) A ( ) b + A x α Dividing by x gives an upper bound on the relative error. x x A ( ) b α x + A = α A ( b x + A ) α A ( b b + A ) Like Theorem 2.0 did for Theorem 2.5, we can give a weaker result that is easier to actually use. Theorem 2.5. Let Ax = b where A is an invertible matrix and b, x are nonzero. Let A x = b, and define A A, b = b b, x = x x. Let α = ( A) (A). If α < then: x x ( b α c(a) b + A ) Proof. exercise. [ Problem 2.6. Let, b =. Let A be a matrix with 0 s in the first column and 0 2] numbers of absolute value at most in the second column. Suppose further that b is a vector with numbers of absolute value at most 0. in each position. We want to solve A x = b with A = A+ A, b = b + b, x = x + x. Find the relative error x / x and x / x. exercises Let a) Find a matrix P with I + P. b) Using Theorem 2. with the -norm, can we conclude that A is invertible? If yes, give values a and b such that a A b. c) Using Theorem 2. with the -norm, can we conclude that A is invertible? If yes, give values a and b such that a A b. 2. Let A and R be n n matrices with A invertible, and fix some norm. a) Give a condition on the scalar p (using the norm) such that A + pr is invertible. b) Show that there is a q > 0 such that A + pr is invertible for every p with p < q. 3. a) Let A be an n n matrix, such that the diagonal elements all fall in the range (0.7,.2) and the off-diagonal elements all fall in the range [ 2n, + 2n ]. Consider the matrix P 92

6 such that I +P. Show that P <. What does this tell us about the invertibility of A? b) Does there exist an invertible matrix B, such that B = I +P for some P with P? Either prove that this never happens or give an example. 2 /2 /2 / Let , and you are also given that A = /2 /2 /2 The 4 4 matrix R has all entries in the range ( /24, /24). a) Find the -norms of A and A, and give an upper bound on the -norm of R. b) Find the -norms of A and A, and give an upper bound on the -norm of R. c) Based on the -norms of the various matrices, can you conclude that A+R is invertible? Explain. d) Based on the -norms of the various matrices, can you conclude that A+R is invertible? Explain. e) Is A + R invertible? Explain. 5. Let A be an invertible matrix. Let R = ka for any scalar k. a) Show that A + R is invertible. b) Show that A R = RA = k, and so can take on any positive real value. c) We see that A + R is invertible even though A R = RA can be greater than. Does this contradict Theorem 2.5? Explain. 6. (slightly ) Find examples of invertible matrices A and matrices R such that A and A+R are both invertible, but A R and RA are arbitrarily large. You can do this for specific norms; it is not necessary that each example works for every norm. Explain why this does not contradict Theorem a) Let Ax = b with A invertible, x 0, but b = 0. Adapt the proof of Theorem 2.4 to give a bound on x / x. b) Let Ax = b with A invertible and x = 0 (note that this implies that b = 0). Adapt the proof of Theorem 2.4 to give a bound on x. 8. Consider the matrix A from Exercise 2.4, and let b = [ ] T [, x = ] T ; so Ax = b. Assume that all entries in A and b are in the range ( /24, /24). a) Using the -norm, what can you conclude about x that solves A x = b? b) Using the -norm, what can you conclude about x that solves A x = b? 9. c) Based on the above, what can you conclude about x that solves A x = b? a) Solve x = [ b) Solve x =..000 ] c) Are your solutions similar? Compute the condition number of the matrix. Thinking of the second system as a perturbed version of the first system, what does Theorem 2.5 and Theorem 2.0 say here? correction: Theorem 2.4 and Theorem Recall the functions f(a) = m i= j= n A ij g(a) = max A ij : i m, j n a) If you didn t already do so, show (using the definition) that the functions f and g are indeed norms on the vector space M m,n. 93

7 b) Consider the matrix of Example 2.3. What do f and g say about this matrix using Theorem 2.?. For any ɛ > 0 give a matrix P such that det(p ) < ɛ yet I + P is not invertible. This shows that we can t use determinants instead of norms in Theorem 2.. It also shows that det( ) is not a norm (but you knew that already, right?). 2. a) Show that for P = I that P = for every norm, yet I + P is not invertible. b) Give a matrix P I such that P p = for each p {,, 2}, yet I + P is not invertible. c) Conclude that we cannot strengthen Theorem 2. by replacing P < with P. What happens in the (silly) case of matrices? 3. a) Show that for R = A that RA = A R = for every norm, yet A + R is not b) invertible. For any invertible matrix A, give a matrix R A such that either RA p = or A R p = for each p {,, 2}, yet A + R is not invertible. c) Conclude that we cannot strengthen Theorem 2.5 by replacing P < with P. What happens in the (silly) case of matrices? 4. a) Explain briefly why for two matrices M and N we have MN M N. (This was implicitly mentioned in an earlier chapter but should have been asked explicitly.) b) Prove Lemma 2.2. c) For each of the following matrices, and for each of the -norm, -norm, 2-norm, determine whether c(a) =. [ 0 ] d) For each of the following matrices, and for each of the -norm, -norm, 2-norm, determine the values of a, b, c, d that make c(a) =. a 0 0 b 0 d c 0 5. a) Show that if R A < then R < (Exercise 2.4 might help). b) Looking at the proof of Theorem 2.0, can we now conclude that if R < then A + R is invertible? (Note that it is easier to check R < than R A <, since we don t need to find A, so this would be a somewhat useful criteria.) c) (slightly ) Give examples of matrices A and R such that A is invertible and R < but A + R is not invertible. You can do this for specific norms; it is not necessary that each example works for every norm. 94