MATH0328: Numerical Linear Algebra Homework 3 SOLUTIONS

Transcription

1 MATH038: Numerical Linear Algebra Homework 3 SOLUTIONS Due Wednesday, March 8 Instructions Complete the following problems. Show all work. Only use Matlab on problems that are marked with (MATLAB). Include a published pdf of your Matlab script. There is a template m-file available on the course website for problems 7-9. Solutions to other problems should be written by hand or typeset in L A TEX. Problems 1. (text p.51 Exercise 113) Prove that if, is an inner product, then x =, is a norm. (Hint: You are proving Proposition 108. Using definition 107, verify that the three properties of Definition 103 hold.) Proof. We verify that the properties of Definition 103 hold for =,. Let x, y R N. (i) Definiteness: By the definiteness property in definition 107, x, x 0. Thus x x = x, x 0. If x = 0, then x = 0, which implies x, x = 0. By definition 107, this hold if and only if x = 0. Thus definiteness holds for this induced norm. (ii) Homogeneity: Let α R. Using the bilinearity of <, > given in definition 107, α x =< α x, α x > = α < x, α x > = α < x, x > = α x. 1

2 Taking square root of both sides, we have α x = α x, therefore homogeneity holds. (iii) Triangle Inequality: Assume that x and y are nonzero vectors (otherwise the property follows immediately). Then x + y =< x + y, x + y > =< x, x + y > + < y, x + y > (Defn 107: bilinearity) =< x, x > + < x, y > + < y, x > + < y, y >) (Defn 107: bilinearity) = x + < x, y > + y (Defn 107: symmetry) x + x y + y = ( x + y ) The triangle inequality follows after taking the square root of both sides. Therefore, because satisfies all properties of the Definition 103, it is a norm. The Cauchy-Schwarz inequality for inner products is the following: x, y x, x y, y (Cauchy-Schwarz Inequality). (text p.51 Exercise 116) The unit ball is { x : x 1}. Sketch the unit ball in R for the 1, and norms. Note that the only ball that looks ball-like is the one for the norm. Sketch the unit ball in the weighted norm induced by the inner product x, y := (1/4)x 1 y 1 + (1/9)x y. 1-norm: Let x = [x, y] T, then x 1 = x + y. This means that the boundary of the unit ball, x 1 = 1 is given by the set {(x, y) : x + y = 1}. In other words, y = 1 x, so 1 x 1 and (x, y) satisfy either y = 1 x (for y 0) or y = x 1 (for y 0). Putting together this information produces the diamond below. -norm: This is the usual notion of distance from the origin and gives us the unit circle centered at (0, 0). -norm: Let x = [x, y] T. Then x = max{ x, y }. Setting x 1 gives the boundaries 1 x 1 and 1 y 1. These boundaries produce the square centered at (0, 0) with sides of length.

3 weighted -norm: In this case, x, = 1/4x + 1/9y. This means that the boundary of the unit ball in this norm is the ellipse: 1 = 1/4x + 1/9y. Hence, in the weighted norm, the unit ball is distorted to an ellipse. 3. (text p.56 Exercise 14) Calculate the 1,, and norms of each matrix (do not use MATLAB, complete by hand): [ ] 1 3 A 1 =, A = 4 7 [ Recall that the 1-norm is the max column sum of a matrix (in absolute value) and the -norm is the max row sum of a matrix (in absolute value). Therefore ]. A 1 1 = max { 1 + 4, } = max {5, 10} = 10 A 1 = max { 1 + 3, } = max {4, 11} = 11 A 1 = max { 1 + 3, } = max {4, 10} = 10 A = max { 1 + 3, } = max {4, 10} = 10 As for the -norm, notice that since A is symmetric, we may use Theorem 17: A = max { λ : λ is an eigenvalue of A }. p(λ) = det(a λi) = λ 8λ λ = 8 ± = 8 ± 7 = 4 ± 3 Therefore A As for A 1, since A 1 is not symmetric, we use bullet point four of Remark 11: A 1 is the largest singular value of A 1. This is equivalent to finding the square root of the largest 3

4 eigenvalue of A T 1 A 1. p(λ) = det(a T 1 A 1 λi) = det Therefore A = ([ ]) = λ 75λ λ = 75 ± = 75 ± (text p.56 exercise 15) Show that an orthogonal change of variables preserves the norm. In other words, prove the statement: If O T O = I then O x = x. Proof. Suppose that O represents an orthogonal change of variables. Then, because O T O = I and by the properties of the Euclidean inner product (dot product), for any x R N O x = (Ox) T Ox (defn of -norm) = x T O T Ox (properties of transpose) = x T Ix (O is orthogonal) = x T x (properties of I, identity matrix) = x. (defn of -norm) Hence an orthogonal change of variables preserves the lengths of vectors (the -norm). 5. (text p.56 Exercise 16) Prove that, for an induced matrix norm, I = 1, and A 1 1/ A, where <, > is an inner product (so A = A x max x R N x ). \{ 0} Proof. Let <, > be an inner product. Then, since Ix = x, I = I x x max = max = max 1 = 1. x R N \{ 0} x x R N \{ 0} x x R N \{ 0} 4

5 Let A be a nonsingular matrix. Then, using the property proven above as well as the submultiplicative property of matrix norms, we have 1 = I = A 1 A A 1 A. Rearranging, this gives A 1 1/ A. 6. (text p.59 Exercise 131) If is a vector norm and U an N N nonsingular matrix, show that x := U x is a vector norm (Show that x satisfies the three properties of Definition 103 ). When =, find a formula for the matrix norm induced by. Proof. Let be a vector norm and U a nonsingular matrix. We show that x = U x is, in fact, a vector norm because it satisfies the properties of definition 103. Let x, y R N (i) Definiteness: By the definiteness of the -norm, for any x R, x = U x 0 since is a vector norm. Suppose x = 0. This occurs when U x = 0, which, because is a vector norm, happens if and only if that U x = 0. Because U is nonsingular, U x = 0 if and only if x = 0 (the null space consists only of the zero vector). Hence, definiteness holds for the norm. (ii) Homogeneity: Let α R. Then α x = αu x = α U x (by homogeniety of -norm) = α x. (iii) Triangle Inequality: x + y = U( x + y) (definition of *-norm) = U x + U y (properties of matrix-vector mult) U x + U y (Triangle inequality for -norm) = x + y (definition of *-norm) This proves that is, in fact, a vector norm. Next we find a formula for the induced matrix norm. By definition, the induced norm is A = A x max x R N \{ 0} x UA x = max x R N \{ 0} U x 5

6 Recall that U is nonsingular. This means that both U and U 1 are bijective linear maps on R n. This means that the range of U and U 1 is all of R n. Thus, for every x R n, there is a y R n such that x = U 1 y (or, Ux = y). Hence UA x UA(U 1 y) max = max x R N \{ 0} U x y R N \{ 0} U(U 1 y) UAU 1 y = max y R N \{ 0} y = UAU 1 Therefore, the corresponding induced matrix norm is A = UAU 1. Recall that B = UAU 1 is similar to A. We can think of B as equivalent to matrix A under the change of base given by U (the columns of U would represent the new basis). This matrix norm is the -norm of the A matrix under the change of basis given by U. 7. (MATLAB) Complete the following. (a) Use the matlab commands max, sum, sqrt, and abs to calculate the -norm, 1-norm, and -norm of the following vectors. You may find.-arithmetic helpful: Array vs. Matrix Operations. (b) Verify that your calculations are correct by using the Matlab command norm. Display your results in a table. 1 1e-17 π -1 v 1 = 0 3, v 00 = - 1e4, v π/ 3 = π/4 π/ π/16 clear all; close all; Create variables v1 = [1; 01; 0; 3; -17]; v = [1e-17; 00; -; 1e4; 0]; v3 = [pi; pi/; pi/4; pi/8; pi/16]; 6

7 Calculate norms using max, sum, abs. Recommended to... store these norm values as column vectors. v1mynorms = [max(abs(v1)); sum(abs(v1)); sqrt(sum(v1.ˆ))]; vmynorms =[max(abs(v)); sum(abs(v)); sqrt(sum(v.ˆ))]; v3mynorms = [max(abs(v3)); sum(abs(v3)); sqrt(sum(v3.ˆ))]; Calculate norms using norm command. Recommended to... store these norm values as column vectors. v1norms = [norm(v1,'inf'); norm(v1, 1); norm(v1)]; vnorms = [norm(v,'inf'); norm(v, 1); norm(v)]; v3norms = [norm(v3,'inf'); norm(v3, 1); norm(v3)]; Create table (make sure when publishing the table fits... on the page) mynorms = table(v1mynorms, vmynorms, v3mynorms,... 'RowNames', {'inf', '1', ''}) Norms = table(v1norms, vnorms, v3norms, 'RowNames',... {'inf','1',''}) mynorms = v1mynorms vmynorms v3mynorms inf Norms = v1norms vnorms v3norms inf (MATLAB) Complete the following (without using a for-loop). (a) (MATLAB) Use the matlab commands max, sum, and abs to calculate the -norm, 1-norm, and Frobenius norm of the matrix A given below. (Complete without using a for-loop.) You may find.-arithmetic helpful: Array vs. Matrix Operations. 7

8 (b) Verify that your calculations are correct by using the Matlab command norm. Display your results in a table A = A = [ ; ; ; ; ]; calculate inf norm of A using methods outlined in a and b; AinfNorm = [max(sum(abs(a),)); norm(a,'inf')]; calculate 1 norm of A using methods outlined in a and b... and suppress; AoneNorm = [max(sum(abs(a),1)); norm(a,1)]; calculate Frobenius norm of A using methods outlined in... a and b and suppress; AfroNorm = [sqrt(sum(sum(a.*a))); norm(a,'fro')]; display results in table matrixnorms = table(ainfnorm, AoneNorm, AfroNorm,... 'RowNames', {'mynorm', 'norm'}) matrixnorms = AinfNorm AoneNorm AfroNorm mynorm norm (MATLAB) Complete the following (without using a for-loop). (a) Calculate the eigenvalues of the matrix A in the previous problem using the Matlab command eig. (Type help eig for more on how to use this command.) (b) Calculate the singular values of A (these are the square roots of the eigenvalues of A T A). 8

9 (c) Verify that the -norm of A, given in Matlab by norm(a), is the square root of the largest eigenvalue of A T A. Notice that it does not equal the Frobenius norm of A. eigena = eig(a) singa = eig(a'*a).ˆ(1/) norma = max(singa) norma = norm(a,) eigena = i i i i i singa = norma = norma =